Human-computer interface (HCI)
HCI is not about reading one’s thoughts, though one may think so after watching the many movies that use this idea in various ways. It’s not about visualization of thoughts or controlling one’s behavior either - as much as we are scared of the “Big Brother” watching us. That won’t be a problem, at least in the coming 15-20 years. HCI are devices that are currently used for recording the activity of the brain and other biological systems, which are mostly used in medicine for creating mind-controlled prosthetics or implants for people with particular diseases. It’s worth mentioning that the approaches to HCIs in Europe and the USA are quite different. In Europe, they believe that such devices have to be non-invasive, i.e. wearable, while the US researchers are focusing on implants.
Why use HCI in art
Nowadays, scientists have an opportunity to record the signals coming from the brain and the somatic nervous system as particular output data that can be used in different media installations. Few people know, but one of the first installations that made use of human emotions was created in 1996 by artist Pavel Smetana and was called “the room of desire”. Its idea was based on Tarkovsky's movie “Stalker” where the characters entered a room that fulfilled desires. In the artist’s project, people entered a room with a display, and put on sensors that measured their brain activity and heart rhythm. Those were used to create the content shown on a screen. One of the visitors even ran out of the room and started asking how exactly does the artist know what he’s thinking about.
Speaking about BioArt, modern scientists also use such techniques to express their ideas. For example, the Amygdala project by Marco Donnarumma is a robotic hand that slicing its own skin. Russian media artist Yuri Didevich has recently staged the Neurointegrum performance at the New Stage of Alexandrinsky Theatre, where visual images induced by the brain activity of a single actor were streamed on a large display.
Renowned futurologist Ray Kurzweil predicted that technology will develop faster than humans can comprehend, which is the point of singularity. Everything changes, and sometimes we can’t tell what’s happening, as we can’t be in all places at the same time. In their artwork, media artists are trying to show that becoming cyborgs may be our only way of overcoming singularity.
Interdisciplinary projects
The European Commission has recently launched the STARTS (Science+Technology and Arts) project that is aimed at developing collaboration between scientists and media artists. This collaboration’s goal is to create innovative technologies; several projects are already being developed as part of this initiative. Multiple hackathons on applying human-computer interfaces in different fields are now organized all over the world. Hackathons are a great format that helps bring together people from different fields and motivate them to work together and implement new ideas. Still, the mere three days that most hackathons last are not enough to create something really significant. This is why it is the long-lasting projects that we want to focus on.
Demultiplexia
One of such projects is the experimental performance Demultiplexia that was staged as part of Estonia's 100th anniversary celebrations. The dancers’ brain activity was visualized as different images translated on a display. The idea was to make the dancers communicate between each other by using these images. The images were chosen by a computer based on the dancers’ emotions. This way, we wanted to show the dancers as cyborgs who can communicate without words.
When working on this project, we drew on the fact that the brain’s activity remains the same whether you actually move or just imagine movements. We first decided to record the brain’s activity when one performs choreographic moves and not just simple actions. Still, we couldn’t accomplish this, as it turned out that we can’t record the necessary signals. So, we focused on the dancers’ psychosomatic states: we asked them to “dance” different emotions: fear, joy, solitude, anger, anxiety, etc. Then, we bound particular images to different types of signals. As result, the dancers tried to communicate different emotions, which were also visualized on the screen.
On interactions between scientists and artists
What did we learn from staging Demultiplexia and doing several other interdisciplinary projects? First of all, different people perceive the same things differently: we don’t have a common system for assessing objects and phenomena. For instance, an apple can induce completely different emotions in different people. Secondly, mutual respect is most important for continuing such studies. And that doesn’t have to do with the relationship between scientists and artists only. I believe that oftentimes mathematicians don’t consider sociologists real scientists or vice-versa. Thirdly, working in the field of neurotechnology and neurosomatic systems opens up new opportunities and poses lots of questions that we have to solve here and now. For instance, if you’re working with neurotheater, you have to consider the specifics of theatrical art and also think of ways to translate people’s movements and emotions on a screen. Those are complex questions that imply cooperation.
We publish the results of our project as articles in prestigious journals such as Nature. Neurotechnologies applied in art, neurotheater included, are the instrument that can help us better understand the workings of the human mind. I believe that neurotheater offers an opportunity to create a “collective mind”, i.e. unite the emotions of multiple people to produce a particular image. Or, as some of my colleagues believe, neurotechnology will help break the barriers between humans and computers.
Aleksander Väljamäe’s lecture opened the “Experimental Interaction Design: Physiological Computing Technologies for Performative Arts” international summer school. The school is conducted with support from the School of Digital Technologies of the Tallinn University, New Stage of Alexandrinsky Theater’s New Media Laboratory, and ITMO’s Higher School of Lighting Design.
