Experimental Writing through Human-Scale Natural Language Models

September 19, 2024 Summer 2024

The Workshop

In Human-Scale Natural Language Processing taught by Allison Parrish we participated in an experimental writing workshop that centered on collaborating with ourselves and each other through the creation of a class-sized AI model of our own writing. Practically this meant that every student contributed ~ 1000 words of writing to a class corpus, and we learned how to generate and manipulate text with that corpus as our base. Woven into the class were interactive Jupyter notebooks that served as code lessons, critical conversations of AI, and sharing of each other’s poems and generative writings.

Collage at the Heart of it All

Allison started the class with a presentation of her talk Languages Models Can Only Write Ransom Notes, laying out the argument that “all automated text composition is a form of collage”  to set the tone of the work we’d be doing. Within the presentation she left a collection of citations collaging the language of poets with the work of academics. Our critical study of modern AI models started here through dissecting what the models are actually doing, placing them in a lineage of collage and computational methods before them.

Learning to Cut and Paste in Code

The class worked with Jupyter notebooks to learn methods of filtering, transforming, and cutting up source texts in Python. We learned about the inner workings of older forms of text generators like Markov models and also explored libraries to create work with these models. Learning about vectors, especially interpolation, sounded like making poetry: connecting points in interconnected spaces.

We also labeled the grammar of hundreds of words to build hands-on experience with the processes that construct modern AI models and insert our own authorship onto the practices. Once the corpus was labeled, we explored Tracery grammars and got to create generative text built atop grammar systems.

Our Corpus: bodies of text from the student body

While AI models scrape the whole internet non-consensually of all the text they can find to fuel them, ours was made of contributions to a google doc ranging from poems and journal entries to essays and larger writing endeavors. Our corpus was full of love notes, observations, experiments and sonic textures. Student work (pulling from this corpus) often left participants sounding off in the comments as they saw parts of their own voice in each other’s generative works.