AI & Art Met

AI and Art

art, machine learning

I am reflecting on the AI and Art event put on by the met. Most of the people there were MSFT, Google, MIT, and Cornell. From a cynical point of view, there is a great interest in ML/MI (Machine Learning/ Machine Intelligence) from the industrial-educational complex because it will eventually drive more sales of more powerful computers and software, and because it will increase the need for more education.

From a less cynical perspective, here are my reflections. But first a recap. The Metropolitan Museum of Art is in the process of putting its images (and eventually objects via 3d renderings) on line via wikimedia and wikidata through its open access initiative.  The next question then is how to use this data – ie justify the cost of putting all this stuff online. What is the OKR/KPM/the METRICS? Which is why we enlist google, hackathons, et al.

Now, let me reflect!

Getting it wrong

Two of the panelists: Eva Kozanecka and Serge Belongie, mentioned that they entered the field because they were concerned that the conclusions of ML/MI were incorrect. In particular, as a curator, Eva saw historical connections deduced by ML/MI that were not accurate. This is a powerful driver. The fears with ML I experience bias (ML/MI not recognizing faces with darker pigmentation), or and delusion such as deep fakes (imitating a real person and then putting them in a false situation that appears real).

So, question 1, how does opening up ML/MI to artists or to the public at large mitigate the fear of getting it wrong? Is it merely an example of demystifying the tools. If that is the case, is this the most effective way measure?

Computational Space

The artist Matthew Ritchie gave, what I found to be, the most compelling reason for wanting to work in ML/MI. It allows “computational space to become legible”. There is something to notice here. We are talking about computational space, not digital space or information space. If we imagine the pictorial plane perhaps allows the unconscious to become legible, and the writing plane to allow time/history to become legible, and video/film/production, or interface design  in general that allows the digital space to become legible. Digital is what can be endlessly recontextualized, and infact must be contextualized (ie produced) in order to become meaningful.  The question Ritchie asks (or answers) is what allows the computational space to be legible? What is the computational space. By rendering the relationship between different pieces of data Ritchie is rendering this legible. But, machine Learning is one type of computation.

Ritchie also introduced me to the word Semasiographic: communication by signs. And one of his projects was an ML analysis of diagrams to look for underlying or perhaps the ur-language of graphical communication.

The computational space is made of up many spaces and typologies in a concrete way, and I recommend (as I have before on this blog) reading Quantum Computing Since Democritus by Scott Aaronson to learn what this means. Is ML/MI the only way to make this legible. What does making this legible entail? What does computational space enable that other forms of thought and representation do not?

Tools Vs Work and Transparency

Eva made the comment that the artist Ana Ridler considers different machine learning algorithms as different lenses. And her machine learning project at google is intended to explore this. This is fascinating. But I wonder in what sense are we fetishizing the tools over the work. Like how tube paint allowed impressionists to go paint in the country side. I suspect this question is irrelevant, and it is made irrelevant by the age and modes of expression in the contemporary world. That one of the things the technical image did (Flusser), was remove the distinction between tool and object, and that perhaps the final erasure is between tool, object, observer and creator. This again is another topic worth further exploration.

One participant made a fascinating comment: these works do not make any clearer what ML is actually doing. She didd not have an immediate experience of the ML/MI. And this is absolutely true. In some respects, one of the goals of The Met and open access is pedagogical. And perhaps we should not distinguish between pedagogy and art, or communication and art (propaganda and art).  But it was only until I became a filmmaker that I became aware of the mechanics of different film lenses, what they were doing in a film and why they were being used to communicate something or elicit certain responses. So, I wonder if it is impossible to reveal this information without engaging the observer in the creation process. This also refers to a comment about how sometimes it becomes obvious which Gan is used. As people become more used to, and aware of this technology then people will recognize these techniques more (like autotune or reverb – which was actually a comment linking to Gans to guitar pedals).

In terms of user interaction, I wanted to make a brief mention of tagging. That is user tagging images like blue jay. And in fact there was a tagging party at the Met to tag images. But this a point that Serge Belongie spoke passionately about. That none of this would be possible with out the unpaid labor of millions of people tagging things, either in general, or in their specific area of interest. And although this does sometimes result in bad data, it is by and large incredibly helpful. And we should perhaps talk about the ethics, economics, and what not of tagging. Also he made the comment that tagging is long tail, that most people tag the same sets of images that leave the majority of individual images untagged or perhaps tagged by one person.  This will have its own ramifications for machine learning.  I am sort of interesting in thinking about tagging in analogy with other similar activities in history should any exist (maybe translation – but not really,  The notion of pilgrammiges just popped into my head- I am not sure why)

Final Thoughts

What is the salient thing that ML/MI art would address? If such a question can even be asked.

Some thoughts that crossed my mind, was that it turned interpretations (e.g., ideologies), such as history, into the stuff of creative expression (ie art). That is interpretations NOT processes. One of the panelist said, what does the history of art look like, and how do we turn this into art (I thought of Ezra Pound’s Cantos and other written work that quotes from other work)?  What does it mean to talk about history as the source of artistic expression? For me it means interpretation or ideology – a new type of conceptual art.

Let us imagine that ML is one degree beyond the techno image, that it relies on the techno image but is not the techno image. Flusser wrote that the techno images is about contextualization. The artists is now producer, rather than a painter, or writer.  She puts different pixels in different combinations or different interfaces and recontextualize them. They are recombined, sometimes algorithmically, so these techno images can also be explorations of computational space (algorithmic space). I imagine video to be a techno image. So we can imagine a video to be another form of a techno image.

What does ML/MI do to the techno image? It interprets it. ML/MI is the consequence of outsourcing decision making (thanks Nitzan). It is not number crunching or data analysis. Different ML/MI represent different modes of decision making. We can call this interpretation or ideologies or perhaps something else. The space of the techno image, or computation in general, is the space of context, of bracketed models – e.g,  given gravity is 9,8 and there is no friction the baseball will travel x far in y time. The space of ML is the space of ideology. and value systems, it is the space of all models or meta models.  In machine learning we say: this is the data of chemistry, this is the data of astrophysics, we say this is what fitness looks like, this is what a blue jay is. What is a film made with one GAN vs another. What are the world views of each, the ideologies?  We are still in the grips of the techno image when we express this, how do we break out of this?





In the Archives – Binge Reading Janet Malcolm – thanks Elliott


I loved in The Archives – an inside look at some drama that took place in the 80s regarding a man who was going to become the head of the Freud Archives.


Anyway – recap – the drama.

Anna Freud is getting old, all the psychoanalysis are getting old. Then need a young (ie person under the age of 45) to run the institutions of the guru errrr Founder – Freud! Enter Jeff Masson – a charming young scholar who wants the job. Problem – he is a trouble maker. He is looking to poke holes in psychoanalysis. We all know this person -who desires greatness by disruption (Duchamp for example). Perhaps he is a nihilist. Anyway – Jeff finds a letter in the Archives that makes him believe that Freud should not have discarded the seduction theory, and that all of psychoanalysis is a lie. Obviously you cannot have this sort of person running the Freud Archives. So he is ousted, there is a lawsuit, there are other characters. You get the story.

I am reflecting on this for two reasons

  1. Theory of Seduction – I never took a class on psychology although I did take a class where we read a bunch of Freud’s social and political works. Totem and Taboo, Moses and Monotheism (which could probably be called a conspiracy theory).  I think Freud is an excellent writer and I say again and again, he won over Freud and perhaps the others such as Jung, Reich, Adler, and the rest because of this. Jung in particular is a totally idiosyncratic writer. Perhaps Jung’s medium is painting. Don’t get me wrong – I love Jung -but it is what it is.

So theory of seduction. Fantasy matters. The inner life matters and has material impact on the world. This is my interpretation via the archives of Freud’s breakthrough.  Once does not have to actually be seduced to have some sort of neurosis, all one has to do is think he was seduced.

There are arguments in the book that well, being at Auschwitz is surely different from imagining you were at Auschwitz. and I would agree with that. But perhaps the difference is at the level of somatics and trauma. I am thinking of work like The Body Keeps the Score. And perhaps this is different than neurosis. That mental trauma and physical trauma is different that psychology and somatics are different. Anyway the point is that fantasy matters, it has material impact on ones ability to function in the world.  This is fascinating to me.

2. Internal vs external – I am sort of misstating this but its the best I can do in stream of consciousness blog. In the book Janet Malcolm questions whether or not this whole incident took place because of something inside Masson. Did he has some death wish (or failure wish)? Or was it something structural in the events, in the search for a new head of the Archive etc? Was Masson set up for failure no matter what? It is hard to differentiate the internal and the external because his internal state state or desire feeds into the structural events. But the question is, Did he self implode or did the the structure collapse around him?

  • Self implode – at first I was totally convinced that he self imploded. The picture that I got was of a complete narcissist, and someone somewhat out of control. That he was driving rather than driving himself. His desire to make an impact by breaking things apart made it impossible for him to not implode (since imploding is breaking things)
  • Structural collapse – The whole structure of events made it impossible for Masson to succeed. No matter what he would have been ousted from the archives. This may have been correct. The particular incident that led to his firing was not due to his narcissism and I was lead to believe. (This is pitch perfect New Yorker storytelling). Rather, a new york times reporter was going to write an article about these events come hell or high water – and no matter what it would have been damning for Masson. The real crime was talking to journalists – or the structure of collapse was Journalism itself.

There are sometimes situations where it is near impossible to avoid catastrophe. Here I am thinking about institutional racism (or sexism which I have personal experience with). Some of the structures of the events are such that the only hope is to avoid utter catastrophe, but make no bones about it – it will happen.

In this an un analytical stance. That the external world has this importance, that the internal world is impotent.  Well perhaps we can develop our internal world in such a way that we develop our imaginations and realms of possibilities and can see through the impossibilities to avoid the catastrophies that befall us- even when they seem  inevitable.