Strolling Cities: AI, with poetry

Strolling Cities, at the Italian Pavilion at the Biennale, is a journey through Italian cities reinterpreted by Artificial Intelligence nourished by poems and human voices. Because, says the author Mauro Martino, “creativity evolves thanks to the co-creation between people and generative models”

Before talking about the media artist Mauro Martino it helps to look at his work, that you can view here.

If, after doing so, you feel in a strange world, halfway between estrangement and familiarity, then it means that you have got the point. Then it's worth reading on.

Strolling Cities, in the Italian Pavilion at the Venice Biennale, is Martino's latest effort, created with Ingrid Paoletti, a professor at the Milan Polytechnic. They are video installations created by “feeding” Artificial Intelligence not given in the traditional sense of the term but poems and poetic prose dedicated to cities. Strings of names, adjectives, atmospheres made from letters proposed through the human instrument of the voice that the machine has “digested” and used as an input to imagine urban landscapes. In short, AI worked creatively by processing pre-existing video materials and using the order set by the artist as a canvas.

When it comes to AI, it's good to start from the ground up. What exactly do you work on?

The most recent developments in the field of artificial neural networks have introduced a new aesthetic in art and design. We have discovered latent space. A place that, if not indirectly, we cannot access and that we cannot control. My job is to navigate this immense space and understand the shapes and aesthetics that are generated between the hidden layers of deep neural networks. For me, beauty is to be found in these architectures, using different data collections to train the machines. Human creativity evolves thanks to new forms of co-creation between people and generative models.

Take us into your research center

I am part of a team of 200 people and we deal with core AI, the basic research of Artificial Intelligence. One of the areas of study concerns how AI affects and generates aesthetic experiences. AI is often compared to technical innovations such as the camera or film camera, but this is not exactly the case. AI is not an object to be studied or improved, but an area of study that changes continuously. For this reason, having around a team, advanced machinery and a scientific approach is essential to be at the forefront. To relay innovations, you need to be in a constant stream of research.

We come to Strolling Cities, what are the peculiar characteristics?

The premise is that with AI you can do many things, but you will only have true authorship if you are in control of data collection. For Strolling Cities, we followed all three phases: deciding what and how to collect the data, building the model and the user experience.

If, to tell about the cities, we had been satisfied with plundering the internet we would have obtained partial and focused visions on mass tourism. We were lucky enough to have powerful machinery, the collaboration of the Politecnico di Milano and its students. This is the first peculiarity: having carried out all the photographic surveys with machinery connected by the MindEarth software and a gps programmed to not overlap images. The choice of “what” is rarely addressed in AI art, but it is what poetics is based on: choosing what to tell.

We wanted to look for the atmosphere of the cities even if the most famous landmarks are not the protagonists: there is Rome even if there is no Piazza di Spagna. This is because the photo of the Colosseum or San Marco is just as important as the photo of the bar across the street. A sort of democracy to bring out the most recurring patterns, light and perspective. Another possibility given by the curation of data is the choice of the recovery perspective, designed to give the feeling that the city is advancing around you. This effect has never been seen before: it was designed and programmed.

In short, you have created a dramaturgy of data. But how did poetry come into play?

Exactly, and we then combined the word with the aesthetic choices and created the Voice-to-City technology: which allows you to interact with the machine through the voice. To challenge technology we have chosen to use the most metaphorical and ethereal we had at our disposal: poetry. Dalila Colucci, visual poetry researcher at Harvard accompanied us in the selection. From the first tests we understood that we had created something more than a technological artifact: it was possible to enjoy a new experience of visual poetry created together with Artificial Intelligence.

Obviously the fascination of seeing a word become visual is very strong but what makes Strolling Cities unique is that the car follows you, hooks what it has been fed – images of cities – to the words it hears. He codifies a language related to the landscape and generates images on the fly, he has learned how cities are made and invents them together with the user's voice. These are the cities that don't exist, no two will ever be alike.

Can we take an interactive test?

The link I gave you (see the article opening) is a demo where you can manipulate urban landscapes through words.

Can you take us one step closer to the next algorithm?

We have talked to a generative AI and it is the first time this happens. It is already an approach to a science fiction vision. Obviously, the precision with which the generative model follows you in content or in implementing the interaction of the machine with the results already computed are still open challenges. And then the scenario of the body gesture to be integrated opens up. The future is to create cinema, advertising, visual poetry, interior design through the voice. Speaking to see, disengage from the sketch and enchant users, and give yourself the opportunity to live an interactive experience with a level of unpredictability – and surprise – a lot of fun.