Synthetic Self

June 15, 2023

Synthetic Self creates a lifelike video call interface that responds to audience questions and statements through text or verbal inputs. Here are three examples:

Installation: A cozy chair is set up in front of a screen on which a person waits idly. When you sit down they perk up, explaining they are a synthetic copy of the artist M Eifler. There is something not quite right about the face and voice. Maybe you ask them why the artist wanted to copy themselves. In response they tell you a story about M's struggles with intermittent non-verbal episodes or flip through their database of drawings and show you a page from M’s journal.

Performance: On stage you see a person with a microphone and another projected on a large screen. Together they are giving a presentation on the lived experience of chronic illness. They joke and tell stories together and the virtual performer shows images related to the stories. Near the end of the performance the virtual performer comes out as an AI copy.

Virtual Performance: You join a zoom performance. When you arrive you see 2 video performers who look eerily similar. One is labeled M Eifler and the other Synthetic Self. Both performers speak to and interact with the audience, answering questions and speaking to each other. It's quickly apparent all the little ways the living performer and the synthetic one are different but later after the performance is long over they begin to bleed together in your memory.

Prosthetic Memory

January 25, 2020

Prosthetic Memory is an ongoing experiment in self-augmentation combining handmade journals, daily videos, and a custom AI. With the journal laying flat on a desk a camera captures its pages. The AI compares this feed to its model, determines which page is showing, and projects an associated video memory on the desk nearby. Created to replace the artist's long term memory which was damaged by a childhood brain injury, Prosthetic Memory explores questions like:

-When memories are external, mediated, and public what is the difference between how they are experienced by their creator and an audience?

-Do our assumptions, fears, and uses for AI change when data and machine learning models are created by individuals and families at personal, instead of corporate, scale?

-Given our increasing exposure to algorithmic interventions, how do our identities and perceptions shift when we see ourselves and others through that lens?

The first iteration of Prosthetic Memory, captured in the documentation provided, created a bridge between the physical and virtual components of the memory. In upcoming iterations an age-malleable recreation (deep fake) of the artist will appear on a screen above the notebook, reading from its pages, and, using NLP and sentiment analysis, pointing the user to other pages and videos with similar events or feelings.

Version 1.0 was first exhibited at the Recoding CripTech show at SOMA arts in 2020.

The project was awarded a STARTS Prize 2020 Honorary Mention. 

While AI was a hot topic in the art and wider world as we slid into a new decade, it was often conceptualized as distant, strange and uncontrollable. Prosthetic Memory makes it extremely intimate. It is, to quote mathemusician Vi Hart "a chunk of myself I’ve externalized and personified as “other” in order to experiment with my perspective and my desires."

This project was first exhibited at the Recoding CripTech show at SOMA arts in 2020 and awarded a STARTS Prize 2020 Honorary Mention. 

View this post on Instagram

A post shared by M Eilo (@blinkpopshift)

AI Acne

November 21, 2019

MATERIAL

AI Acne is a set of 22 8.5 x 11 inch watercolor paintings on hydrophobic graph paper. The paintings have transparent overlays printed with green ink. Each green circle in the overlays corresponds to a widely used facial recognition algorithm’s 12% confidence that there is a whole face present at the location. Despite humans easily recognizing faces in these paintings 12% was the highest confidence reported by the algorithm.

CONCEPTUAL

More often than not when I’m interacting with friends, co-workers, family, content creators, or politicians its thru screens. This is a simple reality of the late 2010’s, of my geographical and social position, of my race and gender and class and age. What am I seeing when a face is reduced to mathematical representation on a screen? What is the facial detection algorithm seeing? How is representation different for humans and computers?

EXHIBITIONS

The piece was first exhibited at SYSTEM FAILURE at Minnesota Street Projects in San Francisco in 2019, then again at Beyond Embodiment at at the Brand Library & Art Center in Glendale, CA in 2020.

Masking Machine

July 24, 2019

Masking Machine is a wearable AR apparatus and performance which brings audience, artist, and algorithm face to face. It was inspired by a photography series of the same name.

Masking Machine, documented here performed at the YBCA Bay Area Now 8 opening.

“Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks” Published in ICCV 2017

We are Collage. Data, Instagram, and the Future of AI

November 29, 2018

Collage is the language of the moment, but has been for over 100 years. Lets walk through where it came from (Dada), what it’s up to now (Instagram), and why it’s integral to the future of AI (Deep Fakes, GANS, and the ingrained copy).

Yesterday: Dada

While the technique of cut and paste in its simplest form dates back to the invention of paper in China around 200 BC, collage didn’t become an indispensable mode of communication until mass-produced, printed media was adopted as a material by artists in the early 1900's during the very early stages of Modernism. Enter, stage left Hannah Höch.

Höch was a German Dadaist working in the inter-war Wiemar period and on through WWII. Her works poked surreal at the failings of the Wiemar government, her era’s social constrictions and prescribed roles for women, and an ever increasing flood of pop culture. In a fractured and churned up time, Höch constructed her works from fragments of fashion magazines, newspapers, and photographs stating that there were “no limits to the materials available for pictorial collages — above all they can be found in photography, but also in writing and printed matter, even in waste products.” (1) Her work embraced contradictions, and kneaded together the gender, racial, political, and material tensions of the time including jumping headlong into the art vs. craft tribalism that is still pervasive today.

“[Embroidery] is an art and ought to be treated like one … you, craftswomen, modern women, who feel that your spirit is in your work, who are determined to lay claim to your rights (economic and moral), who believe your feet are firmly planted in reality, at least Y-O-U should know that your embroidery work is a documentation of your own era.”

— Hannah Höch, in Embroidery and Lace, 1918

But Höch, and this is one of the reasons I chose to start with her, didn’t just shred this trove of printed materials, she also produced them. For over a decade, as Madeleine Boucher of Artsy.net writes, “By day, Höch designed embroidery patterns for women’s hobby periodicals and illustrated daily magazines, while by night she quite literally ripped those images apart in her artwork. Her work often violently attacks the ideals of femininity and glorified domesticity that those magazines repackaged and popularized as fashionable.” (2)

Collage stakes value in this kind of dual relationship to the omnipresent materials of culture but this stance is double edged. David Banash, author of Collage Culture, argues that the use of ready-made components was central to art making in the 20th century because while it highlights “the possibilities and limits of an inescapable consumer culture, “ and “has the power to be critical of [its] ideology,” collage also “depends on the materials of consumer culture and quite literally re-enacts the exact processes of mass production and consumption.” (3)

Hannah Höch, Untitled (Large hand over woman’s head), 1930

The collage aesthetic in all its material forms not only helped to spark Modernism, it made huge gains in cultural relevance through Postmodernism and beyond. Let’s take a quick side trip thru that history so we are all on the same page about these concepts and periods.

Eras are a bit like stereotypical teenagers rejecting their parents’ ways. Modernism had the Enlightenment to fight against. The Enlightenment, which was from about 1620 to 1800, mostly describes early scientific and anti-monarchy activities happening in Europe. Modernism, which began in the late 1800’s, rejected the Enlightenment’s certainty and reductionism. Postmodernism, which emerged after WWII, cast off Modernism’s objectivity and universality. And today, in a period I like to call the Transist era, we are sloughing off Postmodernism’s irony and anything goes attitude.

“In considering the names that might possibly be used to designate the new era following “postmodernism,” one finds that the prefix “trans” stands out in a special way. The last third of the 20th century developed under the sign of “post,” which signaled the demise of such concepts of modernity as “truth” and “objectivity,”…”originality”… and “sincerity”… All of these concepts are now being reborn in the form of “trans-subjectivity,”… “trans-originality,” “trans-lyricism,” “trans-sentimentality” etc.”

Mikhail Epstein, Russian literary critic, The Place of Postmodernism in Postmodernity (4)

And through all that upheaval, collage, with its devotion to hybridization, liminality, and intersectionality, proved an important and durable tool for art and communication. “OK, OK, so collage is great and useful and has this whole lineage or whatever. What’s your point? You promised memes and AI!”

Today: Instagram (2019)

So while the producer-consumer hybrid was born in the cauldron of photo collage, it has spent the last 100 years chewing its way into every corner of social, political, and artistic life. Even the process of writing this piece is enabled by the collage aesthetic. I’m clipping quotes from online resources, reorganizing my words on the fly, and pasting the product into a publishing platform. Think hip-hop and sampling, think Pride and Prejudice and Zombies, think inspirational speech supercuts, think every meme you’ve ever participated in, think social media platform interface design and development.

If this were a podcast, this is where I would insert the record scratch.

Because it’s not just the images we use to advertise our avatar-selves on Instagram, or even the functionalities and interfaces provided in the app that have taken on the cut-and-paste aesthetic, but the techniques used to make the the software itself. Instagram started out as a fairly simple photo feed but didn’t catch proverbial fire until it began directly copy-pasting features from its competitor Snapchat. Disappearing photos and videos sent to individuals or groups of friends, which later became public ‘Stories,’ face filters, and decoration with animated stickers all originated with Snapchat. (5)

This is not necessarily a value judgement against Instagram’s design practices, (I have plenty of beef with their choice to include intentionally habit-forming functionality, but not with its copying of competitors) but instead is meant to point out that collage is integral at all scales of techno-communication today. And it’s not just big platforms constructing themselves from bits and pieces taken from elsewhere. Software development these days never starts from blank canvas. There’s nearly always a library, API, or component system snagged from zee interonets or internal legacy code. And sure, yeah, maybe I’m stretching the concept of collage well beyond the confines of its original meaning, but it turns out language is made by those who use it. Get off my smoke-smothered sidewalk language police! (The Camp Fire still burns as I write this. Today the AQI in San Francisco is 265.)

Like evolution, which cannot jump from a present body format directly to the perfect ideal but instead must step blindly through a thicket of tiny mutations, technological change tends to improve incrementally as well. But then, we invented this thing called genetic engineering which allowed huge changes in genetic makeup that would not have been possible via evolution alone. Collage is the genetic engineering of technological innovation. Much the way working in diverse groups of people recombines their varied backgrounds and ways of thinking into more wide-ranging possible outcomes, collage expands the boundaries of what’s possible to create by bringing into contact material of various sources and types

Art-based research for technological exploration, the style of research I specialize in and have done at SAP, Y-Combinator Research, and now Microsoft, leans heavily on collage as an inquiry technique. Rather than focus on incremental improvement, art-based research attempts to jump to far off, precarious, and often not-viable-at-present lily pads in order to broaden our menu of innovative avenues. Collage is a skill, an art, and the key to democratizing the future of AI.

Tomorrow: AI

Laura Hoptman, former curator at the Museum of Modern Art in New York, says collage allows us to “to experience information simultaneously… as a horizontal cloud.”(6) Seems an apt description for today’s AI: tools which essentially allow us to grasp our ever increasing sea of information through condensation and representation. AI is collage at scale. It’s predictions and generations are collections of human knowledge, as in labeled sets of cancer screenings, and material culture, as in troves of images, minced into vast arrays of values, scrutinized and recombined into new knowledge and material.

And as AI becomes more commonplace it will become akin to those hobby periodicals and illustrated daily magazines of Höch’s era, like the gifs and memes of today, itself subject to material recombination. This is already happening. Take generative adversarial networks, introduced in 2016 by Ian Goodfellow (7), a Google Brain researcher. GAN’s are a collage of two preexisting kinds of algorithms. It’s a technique used for creating new versions from a set of originals, for example creating an artificial voice from a library of human speech sounds or a new face from a collection of faces. Think of a GAN as a very patient artist and a very discerning buyer. The artist is commissioned to make a portrait and given a large library of source material. Every time the artist finishes a portrait the buyer gives them a score. At first it’s a pretty bad score, but soon, after say a million tries, the score gets much better (not perfect, but pretty close, hair and irises are still pretty tough).

Now because we are talking about AI we have to keep in mind that neither the “artist” nor the “buyer” has any idea what a face or even what an image is. Both break pixel colors up into arrays of values which they then run statistical comparisons on and make adjustments to. An AI, at least by current definition, can’t understand the problem set to it. It can only, very quickly, aggregate thousands of “data-fied” human actions, like labeling an image, or create statistically similar examples of a collection it is provided, like creating an image. That cancer screening AI might get it right more often than any individual doctor but that’s not because it’s “smarter” but because it uses the knowledge of thousands of doctors all collaged together. (Am I oversimplifying how these piles of calculus work? Yes. There are at least 1 trillion articles online which can teach you the basics. Go forth!)

Now Goodfellow is a super-extra-double expert and at the moment that’s what it takes to make a good AI collage. But what if instead of focusing all our effort on creating easier ways for people to code their own versions of established AI (generators, classifiers, anomaly detectors etc.), instead we facilitate wide ranging AI collage. Glue! Scissors! Tools galore! To empower a large number of people, everyone, bakery owners, digital artists, middle schoolers, to craft useful and meaning-making AI, we shouldn’t exclusively focus on ways that make coding novel models more approachable. The lineage of collage shows us that interfaces that give access to existing AI algorithms and let people play around with recombining them in odd and counter-intuitive ways will lead to an explosion in who is creating AI and what fun, weird and maybe even revolutionary things it’s being asked to do. This is how we avoid obfuscation via automation. This is how we deep-fake the meme wars. This is how we win!

Whoa. Turns out art skills are like… useful? Who’da thunk it…

Notes

Hannah Höch, “On Collage,” in Hannah Höch, ed. Dawn Ades, Daniel F. Herrmann (London: Whitechapel Gallery, 2014)
Madeleine Boucher, “Art or Craft?: Hannah Höch’s Collages Embraced the Conflict Between Art and Craft, Dada and Commercialism”, Artsy.net, published 10/14/14, access 11/12/18, https://www.artsy.net/article/madeleineb-art-or-craft-hannah-hochs-collages-embraced
David Banash interviewed by Rick Poynor, “Collage Culture: Nostalgia and Critique,” Design Observer, published 11/11/2013, accessed 11/17/18 https://designobserver.com/feature/collage-culture-nostalgia-and-critique/38187
Mikhail Epstein, “The Place of Postmodernism in Postmodernity,” published 1998, accessed on 11/17/2018, http://www.focusing.org/apm_papers/epstein.html
Karissa Bell, “Instagram is proof shameless copying pays off,” Mashable, published 5/12/18, accessed on 11/17/2018 https://mashable.com/2018/06/12/instagram-proves-copying-is-the-best-strategy/#NysItTmXfaqo
Rachel Wolff, “Cut-and-Paste Culture: The New Collage,” Artnews.com, published 12/12/13, accessed 11/18/2018, http://www.artnews.com/2013/12/12/the-new-collage/
Wikipedia, accessed 11/19/18 https://en.wikipedia.org/wiki/Generative_adversarial_network
Tero Karras, Timo Aila, Samuli Laine, Jaakko Lehtinen, “Progressive Growing Of Gans For Improved Quality, Stability, And Variation,” Published at ICLR 2018, https://arxiv.org/pdf/1710.10196.pdf

Originally Published on Medium

MASKING MACHINE (PHOTOGRAPHY SERIES)

September 24, 2018

The Masking Machine is a still color digital photography series using machine learning to create masks of the artists face. Each image begins life as a simple selfie, the most quotidian, maligned, and rampant photography of our time. Next the images are feed into a landmark detection algorithm which finds the position of my facial features. Using this data the artist applied a layer of digital makeup, contact lenses, or glasses. Once saved the images are fed back into the landmark detection algorithm and the process begins again, over and over, building up eerie distorted masks of the artists face.

Whether working in a dark room, coaxing chemical systems like wet plate collodion or cyanotype, or working with a black box, bartering with the algorithms like landmark detection or style transfer, what is photography if not scientifically processed image-making through a machine? And when done recursively, feeding the same image through that process again and again the machine itself also becomes visible in the images. Like repeatedly taking a pinhole photo of a pinhole photo until nothing remains but the artifacts of the photographic process itself made visible. Our time is caked with filtered images, enlarged eyes, and whitened teeth, all enabled by nearly invisible photographic algorithms. If we are to live with them we also need to recognize and understand them and Masking Machine is here to make them seen.

Update: Masking Machine was printed and debuted at the 2020 show Recoding CripTech at SOMArts.

Performing with Machines

January 28, 2016

In 2016 BlinkPop began experimenting with collaging machine learning outputs to create performance masks. They used style transfer to mix images of their own face with the data converted styles of various artists and played with both using the masks to augment performance (of spoken poetry) and intentionally interfering with the computers ability to apply the masks.

Visually Similar

June 4, 2013

This piece was the artists first piece utilizing AI. Each collage was constructed from images Google’s Visually Similar Algorithm considers similar to the previous collage. The first collage was seeded with an image of Swell/Say (2007)

Example of results when images of Swell/Say (a large yellow textile sculpture) is added to visually similar