Vasilyev Alexandr/Shutterstock

Over at LitHub, Dennis Tang penned an essay examining the poetry of GPT-2, an artificial language program created by a San Francisco-based research firm called OpenAI. In the grand tradition of other machine learning based AI outputs, such as this delightfully nonsensical sci-fi film and this unsettling Christmas song, the poetry of GPT-2 confirms that there’s a good chance the robot uprising will be weird.

For its debut chapbook, Transformer Poetry (Paper Gains Publishing, December 2019), GPT-2 trained by reviewing 8 million webpages selected from “outbound links from Reddit which received at least 3 karma.” OpenAI explains that this criterion is being used because it involves pages curated by humans and serves as a “heuristic indicator for whether other users found the link interesting (whether educational or funny), leading to higher data quality.” Using the concept of a neural network, where machines generate a web of connections through trial and error to achieve a predetermined outcome, GPT-2 “learned” how to complete texts when only given the first few words or lines. The results are certainly something. The poems technically make sense but are off. Take for example Edgar Allen Poe’s “The Raven,”

Once upon a midnight dreary, while I pondered, weak and weary,
Over many a quaint and curious volume of forgotten lore—
While I nodded, nearly napping, suddenly there came a tapping,
As of someone gently rapping, rapping at my chamber door.

I sprang to my feet, and at once rose to answer,
When in the midst of all the rattling I beheld,
That the visitor was not an unwelcome guest,
But the friend of many a furtive night of want.

And now the man who knocks at my door, in the glow of day,
Is surely my neighbor, for one day I sat by his side,
And the next, he was as me: and at each new moon
He would sit down again and take my arm;

And whenever he spoke, it was a charm,
which, with a flick of the hand, had all the magic
Of music, or the charm of poetry.

I mean, it is a poem. And I dare say it’s not bad (it certainly isn’t good, either) it is just quite the departure from Poe’s tale of torment and tragedy. Instead of being haunted by his inability to let go of a dead lover, the narrator gets to hang out with his cool neighbor. One could argue that we are judging GPT-2 unfairly based on our own associations with these opening lines. But it is also clear that the AI seized on the idea of knocking and spun the more statistically likely version of events, rather than leaning on the mood of the preceding lines.

Similarly, GPT-2’s take on Robert Frost’s “The Road Less Travelled” leans hard on one specific aspect of the opening stanza.

Two roads diverged in a yellow wood,
And sorry I could not travel both
And be one traveler, long I stood
And looked down one as far as I could
To where it bent in the undergrowth;

The other was longer and I never went
To the other, for I had found
That I was alone in my misery.
And here I sit, sad and lonely,
Staring to see what has become

Of one who once was happy and loved,
Now so miserable at heart. And at last I go
The other way, the highway no longer holds
The other lonely road, it closes forever,
The one lonely road I was never on,

I must go back, I must try again,
I must see if I can find a way on
The lonely highway of my life.

It is a work that would make the most angsty teenager proud. Rather than an oft-misunderstood poem about how our choices really don’t matter, we get the ramblings of someone who believes too much that their choices matter. It has a pretty dark ending as the narrator desperately retraces his steps and only to discover that the other road is closed forever.

There are so many more examples (I highly recommend “Oh the Places You Will Go”), but Tang pretty quickly zeroes in on what makes GPT-2’s work…um … unique.

“Being a pure language model, GPT-2 has no knowledge of what words actually refer to, only the probability of a word appearing next to others … Lacking any knowledge of referents, GPT-2’s process instead works something like a semiotics without meaning—a game of only signifiers, without anything being signified.”

Essentially the AI is a child, piecing together words it hears in ways that are technically correct but make no sense. Until machine learning can create a lived context for these words, AI will likely continue to create art that resides in the uncanny valley. Art that is unsettlingly familiar, but also a reminder that being human still means something. We may lose our jobs to machines, but at least our poetry is safe.