Augmented Narratives: In Conversation with Tivon Rice
Models For Environmental Literacy
FIBER Festival 2020 will take place from 24–27 September. With the festival theme of Instability we explore new ways of adapting to an age of planetary and societal changes. Leading up to the event, we are investigating the unstable layers of four subdomains: Becoming Unstable, Sensing Place, Radical Streams and Posthuman Territories. Within Posthuman Territories, we aim to investigate how artistic engineering can play a role in developing an inclusive and sustainable technological infrastructure. What invisible conflicts may emerge due to the changing landscape? Over the next few weeks, we begin our unstable journey by speaking with different artists whose work connects to these themes.
Tivon Rice is an artist and educator working across visual culture and technology. Based in Den Haag (NL) and Seattle (US), his work critically explores representation and communication in the context of digital culture and asks: how do we see, inhabit, feel, and talk about these new forms of exchange? How do we approach creativity within the digital? What are the poetics, narratives, and visual languages inherent in new information technologies? And what are the social and environmental impacts of these systems?
These questions are explored through projects incorporating a variety of materials, both real and virtual. With recent films, installations, and A.I. generated narratives, Rice examines the ways contemporary digital culture creates images, and in turn build histories around communities and the physical environment. While much of Rice’s research focuses on emerging technologies, he continuously reevaluates relationships with sculpture, photography, and cinema. His work then incorporates new media to explore how we see and understand a future thoroughly enmeshed in new data/visual/production systems.
On ‘Models For Environmental Literacy’ — Video installation (2020)
Sensor-based technological systems are being deployed to observe, optimize and manage human interactions with natural environments at a planetary scale. Their implementation would make it possible to harvest great amounts of environmental data and calculate optimal ecological management as a prime weapon to fight climate change and ecological destruction. With the rise of artificial intelligence these systems are more and more trained to function autonomously and let humankind work at previous unimaginable dimensions.
A fundamental question within these developments is: do these systems truly understand the complex ecological reality of environments? A.I. systems can only interact with an environment based on the learning data that was given to them. What if the data was lacking crucial ecological, cultural data or philosophical information, to make them behave as inclusive as possible towards non-human agency?
The work Models For Environmental Literacy explores in a speculative way, if a cluster of AI agents is able to create a new understanding of an ecological environment. Three distinct AIs were trained for a screenplay: the SCIENTIST, the PHILOSOPHER, and the AUTHOR. While they each have a distinct voice, vocabulary, tone, and ‘knowledge’, they are all children of the same large Natural Language Processing system. The AI can at times be confident, assertive, insistent, or impassioned, and at times naïve, uncertain, confused, or sedate. With this video-installation Tivon Rice explores the alternative application of AI-systems and opens a debate on how they are trained and biased.
In an interview we spoke with Tivon about the role of A.I. in making sense of our environments, its potential pitfalls as well as reimaginings of our human-nonhuman relationships.
Jarl Schulp: Can you briefly introduce your practice and your newly commissioned work?
Tivon Rice: Yes, over the past few years my strategy for developing visual and text-based projects has begun with three distinct spheres of activity. But those spheres usually start to overlap and bleed into each other as I invite collaborative voices — both human and virtual — into the work.
The first phase has involved fieldwork with such groups as FOaM, Bioart Society, and of-course FIBER. Especially since moving to Europe in 2017, this has given me the opportunity to learn and share ideas about ecological practice with artists, scientists, designers, and others. So, these field trips are meant to collaboratively develop our written, or spatial, or visual environmental literacy, without the augmentations of AI or machine learning.
The next phase is a bit more technical and involves building tools and processing data, either from the field or from the research that I conduct in parallel with these workshops. For example, processing thousands of images to create 3D models of the landscapes we explored, or collecting and training new language models that can describe those landscapes.
Finally, my studio work involves bringing these materials into new visual and narrative forms. With Models For Environmental Literacy this has become a series of short films that don’t necessarily tell a straightforward story, but rather guide the viewer through a series of digital landscapes, and invite a number of different possible readings through the computer generated texts.
JS: What alternative role(s) could the application of AI technologies play in a different understanding of the ecological reality of the planet?
TR: Of course, large-scale, multidimensional analysis of data can provide very powerful tools for comparative and predictive environmental models. That is clear. But rather than telling us how to manage or perceive an environment, I have found these artificial agents can also, in a way, ask us what we know, feel, believe, and understand. So, by providing a spark for thinking about complex systems in new ways, they can invite us back into the critical and creative loop when thinking about the environment.
JS: What are the potentials and pitfalls of languages based AI creation within the fields of design and storytelling?
TR: I see many potentials specifically when working with language-based AI. And through workshops, conferences, and especially the gatherings organized by Google AMI, I’ve been fortunate to meet a number of Artists and Poets who are thinking both creatively and critically about these tools. For example Ross Goodwin’s experiments with a cinematic script-writing A.I. or a published collection of texts written by an augmented car during a road trip from New York to New Orleans. Also Allison Parrish’s data poetry that breaks language down into its essential phonemic elements in order to explore the sound, shape, and meaning of real and imagined words. And just recently, I helped Arita Baaijens and Mark IJzerman (both FIBER contributors) get started on their own Dutch language models, to create a new corpus of knowledge for the Embassy of the North Sea.
With these potentials in mind, I think it is important for artists not to dismiss the ethical implications of working these tools, just because they use them in a creative, or non-commercial, or non-institutional context. So, it is crucial that groups like the Association for Computational Linguistics have adopted a code of ethics surrounding the collection, training, and use of textual data. These guidelines include the imperative to do no harm, to be honest and transparent in the use of computer generated text, and to understand the biases that exist in both pre-trained and fine-tuned language models.
JS: Your project is an alternative cartography of a series of man-made natural environments, like the IJsseloog. Could you introduce these environments and why you specifically chose them?
TR: Well, not all of these landscapes are human-made. The first chapter maps the small islands surrounding Helsinki, which are actually rising twice as fast as the surrounding waters of the Baltic Sea. This is part of a 10,000 year process of post-glacial rebound, so instead of reflecting an uncanny image of human intervention (like the IJsseloog), they present a more paradoxical image of a system operating in geo-historic rather than human timeframes.
But yes, chapters two and three indeed map the direct and indirect effects of engineering on the Dutch landscape. In the case of the IJsseloog, it is arguably the strangest visual landscape in the country — a perfectly circular island, 1 kilometer across, built for the deep storage of industrial toxic sediment. After first seeing this circle on satellite images, then during subsequent field trips to drone-photograph the site, I couldn’t help imagining it as a kind of monumental frame upon, or a portal into the past, present, and future of the local environment. By the way, the construction is almost exactly twice as wide and twice as tall as Smithon’s Spiral Jetty.
In some ways, the IJsseloog is a kind of punctuation mark (like a period or a comma) in the story of the Zuiderzee Works, the land reclamation project that shaped the northern coastline. So, chapter three shifts focus to the southwest of the Netherlands, where the Delta Works creates a different kind of engineered environment. During FIBER’s Cartographies of the Vanishing Now workshop, we traveled to the Volkerak to observe the algae blooms lining the shores, and to discuss the complex cultural, hydrological, and agricultural histories that have deeply impacted that ecosystem.
JS: Did you experience a new understanding of these environments while you were conducting your research? What new insights or knowledge did you obtain?
TR: Yeah, I guess just now I’ve described these environments through my research, conversations, and one particular — perhaps objective — understanding. But there is something about the mapping process, specifically my drone photogrammetry process, that allows me to experience these islands in a very different way. During the field work, I’m there, physically present in the greater environment, but the drone provides a spatial and optical layer of removal. Paradoxically, it also creates a kind of prosthetic vision that gets me closer to the landscape. Here, it is worth mentioning that I never set foot on any of the islands that I mapped during this project.
So then, days or weeks later in my studio, I process all of the visual data from the drone and make photorealistic 3D models of the islands. This creates an additional layer of removal from the real environment, but once again gets me oddly closer, as I can virtually set foot on the island. I gaze over the rocks at ground level, zoom into details, trace a path across the shoreline. For me, through this process, the islands become something more than just the IJsseloog, or a breakwater in the Volkerak, or Själakobben (one of the small islands outside of Helsinki). They become a ground for exploring the fundamental metaphors of an island, or a human-made landscape, or a natural environment.
During the field work, I’m there, physically present in the greater environment, but the drone provides a spatial and optical layer of removal.
JS: With what data (or types of knowledge) are the agents trained to create their personalities?
TR: Simply put, the three models were trained on 500,000 words each — one from eco-science texts, one from eco-philosophy, and one from eco-fiction. What is new for me is that the three Models For Environmental Literacy contain many authors within each category. So, unlike previous projects for which I created models entirely trained on a single author, such as the complete writings of J.G. Ballard, these agents are a kind of chorus — emulating the tone, vocabulary, and knowledge of multiple sources.
What is challenging and perhaps more interesting about these models is that it becomes less clear which voice is guiding the dialogue. Again, in my previous film, it was very easy to read the computer-generated text and say: yes, that indeed sounds like J.G. Ballard. Now that the output represents a complex of many authors, it is more challenging to say: that sounds like Octavia Butler, or that sounds like Ursula K. LeGuin. So now I understand the output as a kind of hybrid language that incorporates many bodies of knowledge. These agents have learned from many different teachers. Just like me.
With this in mind, I must acknowledge that my decision to incorporate a number of authors, philosophers, and scientists in these models creates a distinct kind of bias in the training and resulting texts. These are sources that I know, that I have read, and that I hope to better understand through this project. But they may be different from the sources that you know, the sources that you have read. So, maybe it’s like being a DJ or making a mix-tape, and my curation of the materials is as important as the materials themselves.
JS: Did the AI surprise you in any way with their understanding or misunderstanding of these environments?
TR: Yes and no. There are times when all three speak very clearly about disaster, the anthropocene, and adaptation to new environments. I see that as a somewhat straightforward understanding of the texts they were trained on. But then, I also hear them speaking about the effects of these situations in very personal, or emotional, or even allegorical terms. They get into conversations about anxiety, loneliness, and collective memory, even non-human memory. And at some point the Author AI started singing its responses, at least the structure and repetition of words was more like a song or a prayer.
Another surprise came when the system seemed to be speaking not about the environment but about the subjectivities of its own output. The Author states: “…our words are not only a vehicle for revealing the world to the reader, but also a vehicle for revealing the reader.” And this resonates with me deeply when I think about reading an AI generated story. Rather than trusting a scientist’s findings, a philosopher’s theories, or a novelist’s virtuosity to guide us, the reader is asked to apply their own knowledge, beliefs, and aesthetics to the text. Thus, the text reveals as much about the reader as it does the (AI) writer.
The Author states: “…our words are not only a vehicle for revealing the world to the reader, but also a vehicle for revealing the reader.”
JS: A systems view of the environment implies that it operates in terms of cause and effect; it is a bounded and knowable network. This in turn invokes possibilities of control and manipulation. Does AI have any potential to reveal environments in ways that escape this trapping of systems theory?
TR: I think systems theory can help us describe the complex interdependencies between different systems. But as you say, the potential to reveal new modes of thinking or new pathways to understanding an environment may require a higher-dimensional mapping — of both the boundaries and permeabilities between systems. And I’m grateful to a number of thinkers, and designers, and artists who have helped me see how the structure of neural networks can serve as a model for this kind of non-polarized or non-binary understanding of social, political and indeed environmental systems. One of the founders of Google’s Artists and Machine Intelligence program, Kenric McDowell, expresses this idea very clearly: “When our representation of knowledge is multidimensional, we see multidimensionality everywhere.”
JS: What are your ideas on creating a future language (and giving voice to the non-human)? What does this mean for the field of art and design?
TR: This is both an interesting and problematic discussion surrounding so-called Artificial Intelligence, because I see contradictions in expecting a non-human language to emerge from a system that is both built and trained by humans. But, with that being said, I think there are many steps to take toward the realization of a future language, and one of those steps is simply being open to the possibility of such a thing. So for artists and designers, this is indeed where working creatively with AI can first help us make sense of the human presence in these machines. Then moving forward, can we imagine working with an agent that doesn’t make sense to us or more precisely makes a different kind of sense?
JS: How do you think we can prepare for adapting to an age of planetary and societal changes — what opportunities are open to artistic making and thinking to contribute to this transformation?
TR: Yeah, I suppose following up on the previous question, it is truly the role of speculative design, speculative fiction, and many forms of artistic making to explore how we will live with the changes we see and the changes we can imagine. I’m currently reading the final novel in Liu Cixin’s Three-Body Trilogy, which envisions the many ways we might cope in the present with the knowledge of an alien encounter that will occur in the future. And while many of the scenarios he describes unfold hundreds of years from now, they also creatively explore contemporary and near-future potentials for our relationships with technology, global politics, cultural identity, and our conceptualization of the other.
Interview conducted by: Jarl Schulp
Edited by: Rhian Morris
