CAPTCHA images are an inevitable and tedious part of feature of being online. But whilst they may seem like mindless admin, your clicks are being collected and used to train massive computer vision models which underpin search engines, and driverless car software.

It was recently calculated that internet users collectively spend the equivalent 500 years, every single day, competing CAPTCHAs. This absurd calculation speaks to the staggering scale data required by AI models and the technical complexity of object recognition.

Aesthetically, the images themselves are gloriously depressing. Gathered from cameras mounted on Google’s Waymo cars they depict a bleak and disturbingly wonky urban landscape featuring virtually no people (for privacy reasons). As technology writer Clive Thompson has noted, this uncanny valley of object-subjects, shot at awkward, voyeuristic angles, turns every cube of the images into a tone poem of melancholia.

The strangeness of the CAPTCHA images is a symptom of the fact that they’re taken by machines for machines, and therefore provide a window on the ‘machine eye view’. They visually demonstrate the distinction between human cognition and machine based learning. The model doesn’t ‘see’ the objects, it detects a series of pixels with different light values, and over time statistically associates different patterns with different categories. While it may seem like we are collectively training the model to think like a human, it is in fact more of a two way street, as we’ve all be turned into a massive data annotation engine.

CAPTCHA Poems uses these images as inspiration for a tongue-in-cheek poetic response to this invisible collaboration. In order to fully reflect the machine eye view, they have been written using an autoregressive language model which uses deep learning to produce fluent, human-like text (NB: before chatGPT was released, but the same basic tech).

Trained on vast amounts of user generated content scraped from the internet, the language model generated poetic responses to simple prompts like ‘a poem about crosswalks written by an AI trained on CAPTCHA images’. The resulting poems are at once an echo chamber for the trite human clichés and technological rhetoric found in abundance online, as well as a window onto the limitations of automation.

However, sometimes the responses speak to the heart of the matter, for example when the response was: ‘This poem was written by an AI who had been tasked with creating poetry about “Machine Eye View.” The author draws attention to how machines perceive the data they are given, which can be difficult because of how binary their programming is.’ When placed side by side, the images and poems speak to the looping, reflexive dialogue between human and machine generated knowledge.