The opening ceremony of the MediaIN festival was organized together with the opening of another city festival, GAMMA. The presentation of the event was prepared in cooperation with the platform m_division. One part of the exhibition zone on the new stage of the Alexandrinsky theater displayed the work of Taras Mashtalir whose interactive multimedia installations have been exhibited in different festivals, for example LUMEN EX, ISEA, ZEBRA, Brooklyn International Film Festival, FIFI, Moscow International Book Festival, Electronic Literature Conference (2012-2017), E-Poetry, Interrupt II, Oslo poesifilm, Liberated Words, Tarp and others.
In one of the halls of the new stage of the Alexandrinsky theater visitors could get answers to their secret questions from Pythia, a digital oracle. Taras Mashtalir was working on this art piece for a couple of years with Lev Panovy, a young programmer. The project made its debut in Russia in MARS Centre, a centre for modern art located in Moscow.
Pythia consists of two screens facing each other and a microphone in the middle. Visitors can use the microphone to ask questions from Pythia, and she will respond in a form of poetic singing. The lyrics of the answer will be displayed on the screens. The mechanism behind this art work is based on speech analysis and creating a vocabulary database in real time. Pythia formulates her answers using this database.
“We are right now in a theater, and many visitors have plays in their minds. I have noticed that many of them rather look at someone else asking questions than reveal their secret questions themselves. Some of the questions might be even intimate. We are talking about a quantum effect, how we communicate with this interface. If a visitor has a cynical attitude, Pythia will answer him the same way” tells Taras Mashtalir.
The algorithm for Pythia was written with the help of the programming language Python and it is an open Google API, which is a powerful speech to text algorithm. This algorithm allows the creators to collect the database, explains the media artist.
“Interactive technologies are the next step in the development of theater, where the distance between the performers and the audience is getting smaller. Pythia and Stella are completely silent without visitors, the audience brings them to life” shared Taras Mashtalir.
There are plans to work with Pythia even further. The media artist has contacted different specialists from the Department of Information Systems and Technologies in High-Tech Business of ITMO University to join him. The goal of the cooperation is to study different resources such as neuro networks and machine learning, as well as developing the algorithm and its ability to meet specific needs.
Another project of the artist is a sound sculpture Stella. This sculpture was introduced to the audience in the main hall of the theater.
Stella is a result the collaboration between the media artist and designers and architects of Saint Petersburg (Pavel Pankratov and Vladimir Antoshsenko) and ITMO University.
The project was carried out in the Art & Science base at ITMO University. This new cluster was founded after the initiative from cooperation of different specialists of science and education, culture and art, engineering and business, and it is meant to serve as an innovative and inspirational environment.
Taras has been working on the concept of Sound Sculptures for seven years, he started studying the subject in MIT Media Lab. The sculpture reacts to the smallest movements and creates a harmonic sound series and a color palette, which can be also called the audiovisual portrait of the person.
“The system of sensors analyses spaces within an artifact. Then this data is organized according to Pythagoras laws, and the sound series is created. The color equivalent is based on the scale of electromagnetic waves from the data, which ensures the synthetic experience. It is believed to have healing qualities” the artist explains.
“Studying neuropsychological phenomena with the help of magnet resonance tomography has shown that detecting consonances activates the cerebral cortex less than detecting dissonance, which feels pleasant. The pleasure of the Pythagoras intervals can be explained by the small amount of energy needed to produce them compared to other kinds of intervals. In consonances we find a perfect harmony and the simplicity of proportions. The combination of consonant intervals and mathematical expressions has been considered divine in history.”
According to the artist himself the sound and light content of the art work can be updated infinitely. For example one of the specialties of Stella is a database of 99 names of god from a Sufi prayer, which in this case demonstrates the lyricism of the Russian language. Stella uses these elements together with the gestures that the visitor makes with his hands to create a poetic work and a melodic composition.
Even commercial companies have expressed their interest towards Stella. A clinic is planning to use the interface as a treatment for their patients. They can for example try the interactive sculpture while waiting for their doctor and this way create a comfortable microclimate around them. The media artist believes that when the cognitive dissonances rise to their peak, this kind of devices will be necessary in everyday life.
Both Pythia and Stella are available for purchasing. Taras Mashtalir is currently working with a group of programmers from Moscow to combine the software of the two artworks into one application. This application can be downloaded on any computer or smart phone. The sensor Stella needs is a video camera, and as most of the devices today have cameras, it is easy to transform any of them into an interactive instrument.
“My task is to take these sculptures from galleries and museums to other environments. City sculptures are an essential part of the architectural appearance of the city. Art in public spaces changes and reflects the cultural connotations of the city and its character. Often we don’t pay attention to the harmful noises of the city like sounds of traffic or construction, but the constant background noise like this is actually so called ‘noise pollution’. My research is connected to sound ecology, which studies the influence of sounds as vibrations on our mind and body as a whole.”
Last year the desire to create harmonic sound environments in cities led to an innovational international initiative and testing of the new technology SONICOLOGY. A series of events were organized in the Central Exhibition Hall, which also served as the debut location for the sound sculpture.
"We created a therapeutic oasis around the sound sculpture. There people could take a break from the chaotic city noises and listen to the always evolving musical creation of the sculpture instead. And the same goes with lights. We have a common understanding with the representatives of the Higher School of Lighting Design from ITMO University on the importance of esthetics in the city space”, explains the media artist.
In the opinion of Taras Mashtalir, for universities it is essential to create a space where young scientists can show their potential and develop their ideas. This way completely new forms and interfaces can be discovered. Collaboration with ITMO University aims to initiate a new incubator targeted to those talented alumni of the university, who would otherwise leave Saint Petersburg to develop their ideas. The creation of Stella is a great example of cooperation between artists, architects and programmers from Saint Petersburg.
“I would like to underline the idea of self identification, activating this process with the help of technological interfaces. We are often subject to external influences and it takes a lot of energy and strength of the individual to avoid these obstacles and maintain a dialogue with the inner self. Technology in my mind is instruments and a collection of interfaces that help people to get to know themselves. Stella and Pythia are like digital mirrors”, the media artist sums.
