Kia ora koutou,

References to the killer robots of popular culture are a recurring feature in public discussion of robots and autonomous weapons. In our recent research we’ve observed how Twitter users play on killer robot references, including mentions of Terminator and Black Mirror, when referring to, Spot, Boston Dynamics’ robot dog. In public discussion of autonomous weapons these references occur in texts intended for general audiences, for example news media reports and popular non-fiction books on autonomous weapons, but also in academic, government and activist writing. The coalition of activist groups advocating for regulation of autonomous weapons even appeals to public familiarity with killer robots in their name: the Campaign to Stop Killer Robots.

The Mapping LAWS project is mapping and analysing features of debates about autonomous weapons, including these popular culture references. Robot Dreams is an interface to explore representations of robots in cinematic imagination and draw attention to themes related to war, violence and threat.

Robot Dreams plays a sequence of sentences mentioning robots from a data-set of over 82,000 movie plots created from Wikipedia. As the robotic voices speak each sentence a visualisation of possible related themes appears. It takes over four hours to watch and listen to more than 1800 robot references. This is academic research, but it is intended as an engaging and playful way to think about robots on film.

This post is an introduction to Robot Dreams and some of the ideas behind it. If you haven’t already, please go take a look at Robot Dreams.

Ideas

When thinking about how to use data to communicate with other researchers and the public, the Mapping LAWS project is drawing inspiration from the work of information designer Giorgia Lupi. Lupi’s advocacy for data humanism challenges the standard visual language of data presentation that prioritises simplifying representations of complex phenomena, but which also reduces possibilities for deep engagement and thinking. The design-driven approach she proposes aims to introduce audiences to the complexity of data, to depict relevant context, and open data up for audiences to explore.

Robot Dreams makes use of the ability to animate graphics and play audio via a web interface to look and listen and think about representations of robots on film. By listening to the cold robotic voices reading movie plot excerpts in a sequence we have the opportunity to experience the prevalence of themes in a unique way, to inspect the range of meanings assigned by the machine-assigned themes and explore connections across movies. The visualisation allows us to see the relative prominence of themes related to robots depicted as circles - larger circles indicate more pervasive themes.

Rather than obscuring the imperfections of the data through simplifying quantifications and abstracted views, Lupi suggests:

“We should learn how to include and render the more qualitative and nuanced aspects of data. We should experiment with how to visualize uncertainty, possible errors and imperfections in our data”

Something interesting and unexpected about Robot Dreams is the ability to engage with the data in this qualitative way.

It is relatively common with computer-assisted text analysis to focus on quantifications: in this case, we could have classified sentences to determine the proportion that indicate war and violence, reporting this with an error rate to abstract the imperfections of the data. The unsupervised, exploratory approach I’ve used highlights a range of possible themes. These themes have been extracted using Empath and its default category dictionaries. The audio presentation of each excerpt with a visual representation of associated themes allows us to evaluate the themes assigned, to engage with the nitty gritty of the data, and perceive the range of meanings associated with words used in the excerpts.

While this method certainly has its limitations, it is interesting in that it opens up the audience to the range of possible meanings, allowing the very human reaction of thinking about and arguing with the associated themes. The randomness and the sequential presentation of the excerpts also allows new views of the data - to notice new features and make new connections.

Madness

There is a serious point to Robot Dreams - to explore themes related to robots in popular culture and depict the prevalence of war, violence and threat - however, listening to the random sequence of excerpts is fun and perhaps a little silly. That is ok!

So go take a look at Robot Dreams and share it around - I’m interested in your reaction. Email me your thoughts!

— Geoff

Method

Robot Dreams has been built with a bit of Jupyter Notebook data processing and hand-crafting of the web interface. Here is a brief rundown of how this was built:

Using Wikipedia’s API, movie articles were collated by querying for pages featuring the film infobox. Then 82,000 movie page sections with titles like “Plot” or “Synopsis” were downloaded.

These plots were processed with Spacy to segment sentences and any sentence featuring robot* were extracted. This doesn’t include related words (e.g. cyborg, android, mech or drone) or robots with names (e.g. Terminator).

Then Empath was run across the sentences to extract themes. Words may (and often do) trigger multiple themes. Empath’s themes are based on a language model that embeds semantic relationships between words. This language model has been trained by the Empath authors on a data-set of fiction plots, so there is some alignment of the language domain (i.e. fiction and movie plots). To avoid over-emphasising themes cued by robot, robots etc, these words were removed.

The pyttsx3 library was used to convert text to robot speech. The robot voices used were part of the accessibility features of the Windows operating system. While there are more realistic machine-generated voices available, I was interested in voices that sounded machine-like.

Hierarchical clustering was used to group related Empath themes and this influenced the position of the circles. For example, war, fight, kill, and military are semantically related and are grouped in the visualisation. I used scikit-learn for the theme clustering. Using a word2vec model via Gensim, vector representations of words within each theme were averaged and then ‘close’ themes grouped.

The dream - the collection of circles indicating the prevalence of different themes - is a packed circle chart. Plotly, which allows lots of control of output and layout, doesn’t produce packed circle charts. However, the circlify library allows calculating a packed circle chart layout. I’ve shared a notebook to show how packed circle charts can be drawn with Plotly.

The web interface was hand-crafted (with love), using SVG output from Plotly, a robot head image and the animation library anime.js.

Acknowledgement and Attribution:

Thanks to Nicki Cartlidge, a soon-to-be PhD candidate at the University of Canterbury. A conversation with her about her planned PhD project on sonification made me think in new ways about representing the data.

The image of the robot is based on an file from Noun Project by Designfour. I’ve adapted this to make it work for the design and animation.