Why bother taking photographs when you can just hallucinate them?
Imagine being able to dream up a picture, describe it to a computer, and have it delivered to you within seconds. Believe it or not, you and I can do that today. We just need a little help from a friend called DALL-E 2.
You might have already guessed that this has something to do with artificial intelligence (AI). A wave of new apps, including Midjourney, Stable Diffusion, and DALL-E 2, can create images based upon text descriptions entered by a user. Computer scientists call this phenomenon ‘hallucination’. The apps are all based upon ‘generative’ artificial intelligence models.
I recently began experimenting with DALL-E 2, released for public use in beta-version. I shared examples of the system in action in a recent photo-essay on Macfilos. I also provided a gentle introduction to the basics of artificial intelligence for those unfamiliar with this branch of computer science. In this blog I will provide a brief digest of my essay and some examples of DALL-E 2’s output.
DALL-E is a play on the name of the Pixar movie robot WALL-E, tweaked to sound like the surname of the surrealist artist Salvador Dali. DALL-E 2 is the latest version. I put the app through its paces by entering brief descriptions of photographs in my collection. In each case, DALL-E 2 generated four remarkable images within twenty seconds of typing in a text string and hitting return.
Here is a matched pair: on the left, my photograph; on the right, the image DALL-E 2 generated in response to the description: ‘A view down a long, medieval colonnade towards a distant vanishing point in black and white’.
Just to be clear, that computer-hallucinated image did not exist previously and was not patched together from existing images. Vast amounts of data and enormous computing power have enabled DALL-E 2’s underlying algorithm to learn the relationship between images and words that describe them. Pixels in the image were arranged by that algorithm to yield an optimal representation of the text string, based on the text ↔ image dataset used in its training.
Here is its interpretation of the phrase I used to describe another of my photographs (on the left): ‘An abstract image from a brightly colored building with deep shadows’
The final photo pair provides an especially convincing example of the power of DALL-E 2. I recently published an article describing my experience of photographing the work of Catalan architect Antoni Gaudi, and his use of trencadis – multi-colored, broken-tile mosaics – to cover exterior surfaces. The challenge I posed DALL-E 2 was to interpret the description: ‘Looking up at a wall covered in tiles in different shades of blue in the style of Antoni Gaudi’.
Machine learning (ML) is the branch of AI enabling these feats. Machine learning algorithms develop capabilities by learning from information supplied to them, rather than through explicit programming. They learn iteratively as more data is supplied, generating models with more and more predictive power. Shown a chest X-ray, a suitably trained machine learning model can diagnose the presence of disease with high accuracy, and in some cases, more accurately than human pathologists.
In future blogs I will share more about machine learning and generative artificial intelligence. Coming back to the question I posed in the title of this article:
Do I think DALL-E 2 spells the end for photography? No.
Do I think it will have a significant impact on the graphic arts? Yes.
The image at the beginning of this article was generated by DALL-E 2 in response to the prompt: “sunlit, snow-capped mountain surrounded by swirling clouds”. Content creators are already incorporating DALL-E 2 into their workflow by generating images like that on demand, rather than paying someone to produce it.
Please let me know what you think of DALL-E 2’s performance and its implications for the future of image creation.