This week I showed some of the pieces I’ve made so far to fellow students and faculty in the Digital Futures program at OCAD U.
A quick recap: my project asks how intelligent machines would portray their own creation if they didn’t have detailed records of it. What might they imagine if their only knowledge came from the faint impressions left in their neural networks of their early training?
To address this question, I used today’s state of the art machine vision techniques, gathered large collections of categorized images, and trained neural networks (Generative Adversarial Networks, or GANs) to produce original images. I presented the images in a variety of media that could be appropriate to machines making images of their own distant past. Images of the pieces are shown below.
Two of the pieces I showed (Evolition and Corporation) incorporate imagery produced by neural networks, and the third (Heaven Underground) is based on an image dataset used to train neural networks for scene recognition.
I’ve been describing this work with neural network as a collaboration for two reasons. Unlike other artists tools, whether digital or otherwise, the output of the neural network is surprising. That is, it is not predictable using conventional computation techniques or algorithms. Also there is a back-and-forth interaction between me and the neural network. I decide what images to feed the neural network, it “decides” what to make of that input. I decide how to combine the output of one network with the output of another, and feed it to a third network for the final results. This differs from a more basic workflow where the neural network makes images based on generic training data, and the human experimenter selects the “best” ones to show.
This collaboration allows me to better understand the capabilities and limitations of neural networks. They are great with putting something down on a blank canvas, where I am terrible with that. I know how to shape a story with levels of semantic depth, whereas machines generally can’t. Telling a story about machines in the distant future is enriched by my hands-on experience of how they excel and how they stumble, and leads me to imagery I could not have developed on my own.
Today’s neural networks struggle to make compelling images on their own, and are not capable of telling a coherent story (unless it’s very short) without human help. Experimenters and artists often compensate for this limitation of their machine by selecting the most compelling finished image or text snippet from a thousand or more samples created by the network. I experimented with more complicated collaborations with image-making neural networks by combining and re-combining the outputs of several specialized neural networks to build up a story.
I don’t think “behind the scenes” or “how it’s made” material is necessary in an art show, but this was mostly a look at process. Some people had earlier expressed confusion about the various steps involved, along the lines of “which part did you do and which part did the machine do?” so I included a how it’s made chart for Corporation, below.
The bust was generated by Robbie Barrat’s Art DCGAN (link) and the faces were generated using Taehoon Kim’s Tensorflow DCGAN (link). The faces used to build the training dataset were gathered using David Sandberg’s implementation of Facenet (link) and I used letsenhance.io’s SR GAN to increase resolution (link)