I am reflecting on the AI and Art event put on by the met. Most of the people there were MSFT, Google, MIT, and Cornell. From a cynical point of view, there is a great interest in ML/MI (Machine Learning/ Machine Intelligence) from the industrial-educational complex because it will eventually drive more sales of more powerful computers and software, and because it will increase the need for more education.
From a less cynical perspective, here are my reflections. But first a recap. The Metropolitan Museum of Art is in the process of putting its images (and eventually objects via 3d renderings) on line via wikimedia and wikidata through its open access initiative. The next question then is how to use this data – ie justify the cost of putting all this stuff online. What is the OKR/KPM/the METRICS? Which is why we enlist google, hackathons, et al.
Now, let me reflect!
Getting it wrong
Two of the panelists: Eva Kozanecka and Serge Belongie, mentioned that they entered the field because they were concerned that the conclusions of ML/MI were incorrect. In particular, as a curator, Eva saw historical connections deduced by ML/MI that were not accurate. This is a powerful driver. The fears with ML I experience bias (ML/MI not recognizing faces with darker pigmentation), or and delusion such as deep fakes (imitating a real person and then putting them in a false situation that appears real).
So, question 1, how does opening up ML/MI to artists or to the public at large mitigate the fear of getting it wrong? Is it merely an example of demystifying the tools. If that is the case, is this the most effective way measure?
Computational Space
The artist Matthew Ritchie gave, what I found to be, the most compelling reason for wanting to work in ML/MI. It allows “computational space to become legible”. There is something to notice here. We are talking about computational space, not digital space or information space. If we imagine the pictorial plane perhaps allows the unconscious to become legible, and the writing plane to allow time/history to become legible, and video/film/production, or interface design in general that allows the digital space to become legible. Digital is what can be endlessly recontextualized, and infact must be contextualized (ie produced) in order to become meaningful. The question Ritchie asks (or answers) is what allows the computational space to be legible? What is the computational space. By rendering the relationship between different pieces of data Ritchie is rendering this legible. But, machine Learning is one type of computation.
Ritchie also introduced me to the word Semasiographic: communication by signs. And one of his projects was an ML analysis of diagrams to look for underlying or perhaps the ur-language of graphical communication.
The computational space is made of up many spaces and typologies in a concrete way, and I recommend (as I have before on this blog) reading Quantum Computing Since Democritus by Scott Aaronson to learn what this means. Is ML/MI the only way to make this legible. What does making this legible entail? What does computational space enable that other forms of thought and representation do not?
Tools Vs Work and Transparency
Eva made the comment that the artist Ana Ridler considers different machine learning algorithms as different lenses. And her machine learning project at google is intended to explore this. This is fascinating. But I wonder in what sense are we fetishizing the tools over the work. Like how tube paint allowed impressionists to go paint in the country side. I suspect this question is irrelevant, and it is made irrelevant by the age and modes of expression in the contemporary world. That one of the things the technical image did (Flusser), was remove the distinction between tool and object, and that perhaps the final erasure is between tool, object, observer and creator. This again is another topic worth further exploration.
One participant made a fascinating comment: these works do not make any clearer what ML is actually doing. She didd not have an immediate experience of the ML/MI. And this is absolutely true. In some respects, one of the goals of The Met and open access is pedagogical. And perhaps we should not distinguish between pedagogy and art, or communication and art (propaganda and art). But it was only until I became a filmmaker that I became aware of the mechanics of different film lenses, what they were doing in a film and why they were being used to communicate something or elicit certain responses. So, I wonder if it is impossible to reveal this information without engaging the observer in the creation process. This also refers to a comment about how sometimes it becomes obvious which Gan is used. As people become more used to, and aware of this technology then people will recognize these techniques more (like autotune or reverb – which was actually a comment linking to Gans to guitar pedals).
In terms of user interaction, I wanted to make a brief mention of tagging. That is user tagging images like blue jay. And in fact there was a tagging party at the Met to tag images. But this a point that Serge Belongie spoke passionately about. That none of this would be possible with out the unpaid labor of millions of people tagging things, either in general, or in their specific area of interest. And although this does sometimes result in bad data, it is by and large incredibly helpful. And we should perhaps talk about the ethics, economics, and what not of tagging. Also he made the comment that tagging is long tail, that most people tag the same sets of images that leave the majority of individual images untagged or perhaps tagged by one person. This will have its own ramifications for machine learning. I am sort of interesting in thinking about tagging in analogy with other similar activities in history should any exist (maybe translation – but not really, The notion of pilgrammiges just popped into my head- I am not sure why)
Final Thoughts
What is the salient thing that ML/MI art would address? If such a question can even be asked.
Some thoughts that crossed my mind, was that it turned interpretations (e.g., ideologies), such as history, into the stuff of creative expression (ie art). That is interpretations NOT processes. One of the panelist said, what does the history of art look like, and how do we turn this into art (I thought of Ezra Pound’s Cantos and other written work that quotes from other work)? What does it mean to talk about history as the source of artistic expression? For me it means interpretation or ideology – a new type of conceptual art.
Let us imagine that ML is one degree beyond the techno image, that it relies on the techno image but is not the techno image. Flusser wrote that the techno images is about contextualization. The artists is now producer, rather than a painter, or writer. She puts different pixels in different combinations or different interfaces and recontextualize them. They are recombined, sometimes algorithmically, so these techno images can also be explorations of computational space (algorithmic space). I imagine video to be a techno image. So we can imagine a video to be another form of a techno image.
What does ML/MI do to the techno image? It interprets it. ML/MI is the consequence of outsourcing decision making (thanks Nitzan). It is not number crunching or data analysis. Different ML/MI represent different modes of decision making. We can call this interpretation or ideologies or perhaps something else. The space of the techno image, or computation in general, is the space of context, of bracketed models – e.g, given gravity is 9,8 and there is no friction the baseball will travel x far in y time. The space of ML is the space of ideology. and value systems, it is the space of all models or meta models. In machine learning we say: this is the data of chemistry, this is the data of astrophysics, we say this is what fitness looks like, this is what a blue jay is. What is a film made with one GAN vs another. What are the world views of each, the ideologies? We are still in the grips of the techno image when we express this, how do we break out of this?
