It’s no secret that all the emotions we experience, including pleasure, fear, anger and so on, are generated in our brain. However, we rarely think about how our gray matter works when we are watching a sad movie or reading a funny comic. But each emotion activates certain areas in our brain and the spectrum of feelings that we all experience every day can be represented in the form of a diagram.
This is exactly what the project conducted by ITMO University students for the Rosphoto exhibition center focuses on. The project was presented in the Art & Science format on the Night of Museums, however, it has still been demonstrated at various IT events. For example, it was included in the program of the 12th Public Relations Russian Student Forum PRKiT 2019, which took place at the beginning of winter.
See your emotions
The project was initiated two years ago when the specialists of the Rosphoto museum and exhibition center approached ITMO University with a proposal to participate in the Night of Museums with a joint project, which would combine art, physiology and virtual reality technologies.
“Every year, we organize a range of hi-tech projects for various St. Petersburg’s museums,” shares Artem Smolin, the project’s coordinator and head of ITMO University’s Center for Usability and Mixed Reality, dean of the Faculty of Software Engineering and Computer Systems. “Our colleagues from the museum invited us to work on an interdisciplinary project, which was conducted by two of our Bachelor’s students majoring in computer technologies in design as part of their theses. We drew information from research articles related to the study of the human brain, to learn about what parts of the brain are activated when people experience various emotions. We talked to the coordinator of the museum and decided to combine our study with the demonstration of archival photographs.”
According to one of the project’s co-authors, ITMO University Master’s student Nadezhda Prokopenko, there were many options proposed for how this idea could be implemented. Eventually, it was decided to create a virtual location with six photographs from the Rosphoto archives, which would house a translucent model of the human brain.
“We picked six photographs for this project: the Milky Way, the besieged St. Petersburg, mother and daughter, an abstract photograph, people playing cards, and people cleaning sculptures in the garden. We interviewed a group of people and asked them to choose which of the eight emotions each image evokes in them. The following emotions were proposed: fear, anger, sadness, joy, admiration, pleasure, tenderness and empathy. Then we selected the three most popular emotions for each photo image. When a user puts on a Samsung gear VR headset, they can see what emotions each photograph evoked among the respondents, and where exactly in the human brain these emotions are generated,” says Nadezhda.
Lost in development
Apart from picking photograms, conducting a survey and creating virtual visualization, it was necessary to integrate the developments into the virtual reality headset. To accomplish this goal, the students had to solve a number of technological problems.
“We had some difficulties with the engine, as we needed to adjust the headset,” explains Nadezhda. “The problem with that was that there was not much information about how to do that in the official documentation. Since the headset needs a smartphone for its work, we had to adapt the engine to the gadget’s performance. When we just started testing our product, it turned out that the frame rate was not quick enough. So when people turned their heads, the picture didn’t quite keep up with their movements. To fix this problem, we had to do retopology, change textures, etc. Then we modified the model of the brain, simplified its geometry a bit.”
The project received good responses from the visitors of the Night of Museums, as well as the administration of Rosphoto and representatives of ITMO University.
Future prospects
“Thanks to this project, I learned a lot about various events and competitions in the field of VR technologies. Now I understand that I’m very much interested in this field and want to further develop in it,” shares Nadezhda Prokopenko.
Though the Night of Museum has long ended, the project is still alive and demonstrated at various events, for example, at the 12th Public Relations Russian Student Forum PRKiT 2019.
“This was meant to be a one-time project”, explains Artem Smolin. “But because it turned out so well, we decided to continue showing it at various events.”
