Collage is the language of the moment, but has been for over 100 years. Lets walk through where it came from (Dada), what it’s up to now (Instagram), and why it’s integral to the future of AI (Deep Fakes, GANS, and the ingrained copy).

Yesterday: Dada

While the technique of cut and paste in its simplest form dates back to the invention of paper in China around 200 BC, collage didn’t become an indispensable mode of communication until mass-produced, printed media was adopted as a material by artists in the early 1900's during the very early stages of Modernism. Enter, stage left Hannah Höch.

Höch was a German Dadaist working in the inter-war Wiemar period and on through WWII. Her works poked surreal at the failings of the Wiemar government, her era’s social constrictions and prescribed roles for women, and an ever increasing flood of pop culture. In a fractured and churned up time, Höch constructed her works from fragments of fashion magazines, newspapers, and photographs stating that there were “no limits to the materials available for pictorial collages — above all they can be found in photography, but also in writing and printed matter, even in waste products.” (1) Her work embraced contradictions, and kneaded together the gender, racial, political, and material tensions of the time including jumping headlong into the art vs. craft tribalism that is still pervasive today.

“[Embroidery] is an art and ought to be treated like one … you, craftswomen, modern women, who feel that your spirit is in your work, who are determined to lay claim to your rights (economic and moral), who believe your feet are firmly planted in reality, at least Y-O-U should know that your embroidery work is a documentation of your own era.”

— Hannah Höch, in Embroidery and Lace, 1918

But Höch, and this is one of the reasons I chose to start with her, didn’t just shred this trove of printed materials, she also produced them. For over a decade, as Madeleine Boucher of Artsy.net writes, “By day, Höch designed embroidery patterns for women’s hobby periodicals and illustrated daily magazines, while by night she quite literally ripped those images apart in her artwork. Her work often violently attacks the ideals of femininity and glorified domesticity that those magazines repackaged and popularized as fashionable.” (2)

Collage stakes value in this kind of dual relationship to the omnipresent materials of culture but this stance is double edged. David Banash, author of Collage Culture, argues that the use of ready-made components was central to art making in the 20th century because while it highlights “the possibilities and limits of an inescapable consumer culture, “ and “has the power to be critical of [its] ideology,” collage also “depends on the materials of consumer culture and quite literally re-enacts the exact processes of mass production and consumption.” (3)

Hannah Höch, Untitled (Large hand over woman’s head), 1930

The collage aesthetic in all its material forms not only helped to spark Modernism, it made huge gains in cultural relevance through Postmodernism and beyond. Let’s take a quick side trip thru that history so we are all on the same page about these concepts and periods.

Eras are a bit like stereotypical teenagers rejecting their parents’ ways. Modernism had the Enlightenment to fight against. The Enlightenment, which was from about 1620 to 1800, mostly describes early scientific and anti-monarchy activities happening in Europe. Modernism, which began in the late 1800’s, rejected the Enlightenment’s certainty and reductionism. Postmodernism, which emerged after WWII, cast off Modernism’s objectivity and universality. And today, in a period I like to call the Transist era, we are sloughing off Postmodernism’s irony and anything goes attitude.

“In considering the names that might possibly be used to designate the new era following “postmodernism,” one finds that the prefix “trans” stands out in a special way. The last third of the 20th century developed under the sign of “post,” which signaled the demise of such concepts of modernity as “truth” and “objectivity,”…”originality”… and “sincerity”… All of these concepts are now being reborn in the form of “trans-subjectivity,”… “trans-originality,” “trans-lyricism,” “trans-sentimentality” etc.”

Mikhail Epstein, Russian literary critic, The Place of Postmodernism in Postmodernity (4)

And through all that upheaval, collage, with its devotion to hybridization, liminality, and intersectionality, proved an important and durable tool for art and communication. “OK, OK, so collage is great and useful and has this whole lineage or whatever. What’s your point? You promised memes and AI!”

Today: Instagram (2019)

So while the producer-consumer hybrid was born in the cauldron of photo collage, it has spent the last 100 years chewing its way into every corner of social, political, and artistic life. Even the process of writing this piece is enabled by the collage aesthetic. I’m clipping quotes from online resources, reorganizing my words on the fly, and pasting the product into a publishing platform. Think hip-hop and sampling, think Pride and Prejudice and Zombies, think inspirational speech supercuts, think every meme you’ve ever participated in, think social media platform interface design and development.

If this were a podcast, this is where I would insert the record scratch.

Because it’s not just the images we use to advertise our avatar-selves on Instagram, or even the functionalities and interfaces provided in the app that have taken on the cut-and-paste aesthetic, but the techniques used to make the the software itself. Instagram started out as a fairly simple photo feed but didn’t catch proverbial fire until it began directly copy-pasting features from its competitor Snapchat. Disappearing photos and videos sent to individuals or groups of friends, which later became public ‘Stories,’ face filters, and decoration with animated stickers all originated with Snapchat. (5)

This is not necessarily a value judgement against Instagram’s design practices, (I have plenty of beef with their choice to include intentionally habit-forming functionality, but not with its copying of competitors) but instead is meant to point out that collage is integral at all scales of techno-communication today. And it’s not just big platforms constructing themselves from bits and pieces taken from elsewhere. Software development these days never starts from blank canvas. There’s nearly always a library, API, or component system snagged from zee interonets or internal legacy code. And sure, yeah, maybe I’m stretching the concept of collage well beyond the confines of its original meaning, but it turns out language is made by those who use it. Get off my smoke-smothered sidewalk language police! (The Camp Fire still burns as I write this. Today the AQI in San Francisco is 265.)

Like evolution, which cannot jump from a present body format directly to the perfect ideal but instead must step blindly through a thicket of tiny mutations, technological change tends to improve incrementally as well. But then, we invented this thing called genetic engineering which allowed huge changes in genetic makeup that would not have been possible via evolution alone. Collage is the genetic engineering of technological innovation. Much the way working in diverse groups of people recombines their varied backgrounds and ways of thinking into more wide-ranging possible outcomes, collage expands the boundaries of what’s possible to create by bringing into contact material of various sources and types

Art-based research for technological exploration, the style of research I specialize in and have done at SAP, Y-Combinator Research, and now Microsoft, leans heavily on collage as an inquiry technique. Rather than focus on incremental improvement, art-based research attempts to jump to far off, precarious, and often not-viable-at-present lily pads in order to broaden our menu of innovative avenues. Collage is a skill, an art, and the key to democratizing the future of AI.

Tomorrow: AI

Laura Hoptman, former curator at the Museum of Modern Art in New York, says collage allows us to “to experience information simultaneously… as a horizontal cloud.”(6) Seems an apt description for today’s AI: tools which essentially allow us to grasp our ever increasing sea of information through condensation and representation. AI is collage at scale. It’s predictions and generations are collections of human knowledge, as in labeled sets of cancer screenings, and material culture, as in troves of images, minced into vast arrays of values, scrutinized and recombined into new knowledge and material.

And as AI becomes more commonplace it will become akin to those hobby periodicals and illustrated daily magazines of Höch’s era, like the gifs and memes of today, itself subject to material recombination. This is already happening. Take generative adversarial networks, introduced in 2016 by Ian Goodfellow (7), a Google Brain researcher. GAN’s are a collage of two preexisting kinds of algorithms. It’s a technique used for creating new versions from a set of originals, for example creating an artificial voice from a library of human speech sounds or a new face from a collection of faces. Think of a GAN as a very patient artist and a very discerning buyer. The artist is commissioned to make a portrait and given a large library of source material. Every time the artist finishes a portrait the buyer gives them a score. At first it’s a pretty bad score, but soon, after say a million tries, the score gets much better (not perfect, but pretty close, hair and irises are still pretty tough).

Now because we are talking about AI we have to keep in mind that neither the “artist” nor the “buyer” has any idea what a face or even what an image is. Both break pixel colors up into arrays of values which they then run statistical comparisons on and make adjustments to. An AI, at least by current definition, can’t understand the problem set to it. It can only, very quickly, aggregate thousands of “data-fied” human actions, like labeling an image, or create statistically similar examples of a collection it is provided, like creating an image. That cancer screening AI might get it right more often than any individual doctor but that’s not because it’s “smarter” but because it uses the knowledge of thousands of doctors all collaged together. (Am I oversimplifying how these piles of calculus work? Yes. There are at least 1 trillion articles online which can teach you the basics. Go forth!)

Now Goodfellow is a super-extra-double expert and at the moment that’s what it takes to make a good AI collage. But what if instead of focusing all our effort on creating easier ways for people to code their own versions of established AI (generators, classifiers, anomaly detectors etc.), instead we facilitate wide ranging AI collage. Glue! Scissors! Tools galore! To empower a large number of people, everyone, bakery owners, digital artists, middle schoolers, to craft useful and meaning-making AI, we shouldn’t exclusively focus on ways that make coding novel models more approachable. The lineage of collage shows us that interfaces that give access to existing AI algorithms and let people play around with recombining them in odd and counter-intuitive ways will lead to an explosion in who is creating AI and what fun, weird and maybe even revolutionary things it’s being asked to do. This is how we avoid obfuscation via automation. This is how we deep-fake the meme wars. This is how we win!

Whoa. Turns out art skills are like… useful? Who’da thunk it…

Notes

Hannah Höch, “On Collage,” in Hannah Höch, ed. Dawn Ades, Daniel F. Herrmann (London: Whitechapel Gallery, 2014)
Madeleine Boucher, “Art or Craft?: Hannah Höch’s Collages Embraced the Conflict Between Art and Craft, Dada and Commercialism”, Artsy.net, published 10/14/14, access 11/12/18, https://www.artsy.net/article/madeleineb-art-or-craft-hannah-hochs-collages-embraced
David Banash interviewed by Rick Poynor, “Collage Culture: Nostalgia and Critique,” Design Observer, published 11/11/2013, accessed 11/17/18 https://designobserver.com/feature/collage-culture-nostalgia-and-critique/38187
Mikhail Epstein, “The Place of Postmodernism in Postmodernity,” published 1998, accessed on 11/17/2018, http://www.focusing.org/apm_papers/epstein.html
Karissa Bell, “Instagram is proof shameless copying pays off,” Mashable, published 5/12/18, accessed on 11/17/2018 https://mashable.com/2018/06/12/instagram-proves-copying-is-the-best-strategy/#NysItTmXfaqo
Rachel Wolff, “Cut-and-Paste Culture: The New Collage,” Artnews.com, published 12/12/13, accessed 11/18/2018, http://www.artnews.com/2013/12/12/the-new-collage/
Wikipedia, accessed 11/19/18 https://en.wikipedia.org/wiki/Generative_adversarial_network
Tero Karras, Timo Aila, Samuli Laine, Jaakko Lehtinen, “Progressive Growing Of Gans For Improved Quality, Stability, And Variation,” Published at ICLR 2018, https://arxiv.org/pdf/1710.10196.pdf

Originally Published on Medium