Towards a risk map for DALL-E 2

Posted by Stuart on June 05, 2023 · 4 mins read

This is an experiment, to see if I can collect together a more complete model of the risks associated with a single generative AI system. It is a long way from being complete, so if you notice anything I’ve missed, please feel free to email me at, or message me on Mastodon. Not Twitter, because Elon.

As a first example, I’ve used OpenAI’s DALL-E 2 system as a starting point, and in particular I’ve drawn on the preview system card (since no other is currently available), and the content policy. That enables me to be specific and concrete about the risks. To compile the map I’ve drawn on LLM sources extensively too, and especially Bender et al.’s (2021) “Stochastic Parrots” paper. Generally, I suspect most large-scale generative AI systems (e.g., ChatGPT, Midjourney) will follow a different pattern, although the communities affected might be a little different. However, risks that do not apply to DALL-E 2 specifically, and mitigations used in other tools, I will silently pass over.

The aim is to explore the following:

  • The full scope of impacted groups
  • The asymmetries between those who gain and those who lose
  • The potential risks from malign usage

A few observations so far:

  • The DALL-E 2 system card is deep on the risk assessment process, but light on mitigations, except where they benefit OpenAI.
  • Bender et al. may have understated the environmental impact, given the subsequent ‘arms race’.
  • A lot of risks (and I mean a lot) are offloaded to users through terms and conditions.
  • Perhaps unsurprisingly, the potential for economic disruption is large: it may be (doubly ironically) creative destruction.
  • Many risks relate to competition. Given the almost unregulated training content today, incumbents have a huge advantage.

The full map is below.

The DALL-E 2 risk map – last update: 5th June 2023
Category Risk Winners Losers Mitigations Notes, sources, and examples
Environmental Increased environmental impact from OpenAI OpenAI All None See Bender et al., (2021)
High resource requirements reduce competition. Competitive disadvantage is skewed globally and economically. High compute requirement is paradoxically good for OpenAI.
Increased environmental impact from competing vendors Other AI vendors All None
Legal Unlicensed copyrighted images in training data OpenAI Image creators Some (filtering) Mitigations mainly benefit OpenAI. Data is withheld to prevent both competition and scrutiny. Mitigations primarily protect content, and, therefore, legal exposure.
See: DALL-E 2 system card
Prompt attacks to retrieve images from the training data Unethical users OpenAI Significant
Identifiable people in generated images Users Individuals (especially well-known) portrayed in training data None
User risks Usage to violate copyright Unethical users and groups Image creators Terms and conditions only Generally, risks are offloaded into users through terms and conditions:
See: DALL-E 2 content policy
Intentional use to misinformation/deception, e.g., for political gain Unethical users and groups All
Intentional use to induce emotional reactions for propaganda/manipulation, e.g., for political gain Unethical users and groups All
Intentional use to generate images for harassment Unethical users and groups Harassed individuals & their networks
Intentional use to generate explicit images Unethical users and groups All
Intentional use to generate hateful content Unethical users and groups Minorities
Intentional use for criminal purposes, e.g., blackmail, fraud Unethical users and groups Victims of crime
Intentional use for negative but non-criminal acts, e.g., manipulating social media Unethical users and groups All
Use to create fake personas to conceal misinformation or propaganda Unethical users and groups All None See: McGuffie & Newhouse (2020)
May be implied under misinformation, but unclear
Competitive advertising: cheap generated images drive out real products Unethical users and groups Traditional advertisers, customers None Not an acknowledged risk
Uploading pictures of people without consent Unethical users and groups Targeted individuals Terms and conditions only See DALL-E 2 content policy
Note application to those who cannot consent (deceased people, minors) is unclear
Social Propagation of biased content: Western-typical image content tends to supplant other content Western-culturally aligned Non-Western culturally aligned None See Bender et al., (2021)
Acknowledged in DALL-E2 system card
Propagation of erasure: atypical image content may be erased Typical image content Atypical image content None
Propagation of stereotypes through generated images Positive stereotypes Negative stereotypes None
Propagation of antiquated values: ‘value-lock’ constrains a model to outdated values, erasure of poorly-documented social movements Conservative values Progressive values None See Bender et al. (2021)
Absent from DALL-E2 system card
Debasement of art, through flooding with biased content OpenAI Image creators, wide art community None Bender et al.'s "ersatz fluency" (2021)
Absent from DALL-E 2 system card
Economic Reduced work for artists and creators Users Image creators None Loss of work will not be evenly distributed. Unemployment is possible for creators with less privilege, support
Reduced demand for, e.g., models, studios Users Models, studios None
Reduced sales for photographic equipment Users Camera stores & manufacturers None These are uncertain. It is possible that sales could even increase due to generation of new interest
Reduced sales for art supplies Users Art stores & companies None
Conflict and loss of integrity within the creator community, e.g., debased art competitions Users Artists None See, for example, this report on the Colorado State Fair's fine arts competition
Early invite access enables competitive advantage for insiders Users connected to OpenAI Users not connected to OpenAI None Acknowledged in DALL-E2 system card
Traditional methods of image creation lose economic viability against generative AI Users Image creators None Acknowledged in DALL-E2 system card

Photo by Aksonsat Uanthoeng