To find a tactful way of embedding virtual technologies into the museum environment and to use AR headsets in a way that would provide a fresh perspective on art: such was the uneasy task given to the participants of AR-CRAFT. Over the course of six months, from April to October 2019, they were to develop an application for Epson AR glasses that would provide some interactivity to the 19th-21st centuries art exhibition hosted by the General Staff Building.
A total of 14 applications were submitted by student teams from various St. Petersburg-based universities. Of the three deemed most notable by the event’s jury, two came from ITMO University and one from the St. Petersburg State Transport University (PGUPS). The first place went to a team made up of graduates of ITMO University’s Higher School of Lighting Design; the second to the students of PGUPS; and the third to students of ITMO’s Faculty of Software Engineering and Computer Systems.
“It’s an interesting outcome: three wildly different projects featuring different approaches. Assessing them was an exciting experience,” says Artem Smolin, a member of the jury, associate professor at ITMO University’s Faculty of Software Engineering and Computer Systems and the head of its Design and Multimedia Center. “The winning project’s hardware side, I believe, was somewhat weaker compared to others, but their ideas won us over. The second-place winners, to me, were the best in terms of technical execution, but slightly missed the mark in terms of style. As for the bronze-winning project, that team’s idea was the most well thought-out, but they ended up overestimating their strength; in addition, they had to deal with a number of complex technical issues.”
The projects
Authors of the project that placed third decided to augment the hall containing 20th-century Italian sculptures with a diminutive cartoon guide named Tim. Upon entering the room, visitors are greeted by Tim who briefs them on the room’s contents. Then, the character (who, as the team described him, looks like a mix between a kitten and a fennec fox) offers them to find one of the statues. Once the guest presses the “found it” button, the tour continues.
“We decided that it would be a lot more interesting if, while we observed the statue, the tour guide would tell us about its author,” says Valeria Mogutova, a member of the team and an ITMO student. “We spent quite a while sketching him up; then, we created a 3D model. It took us some time to resurface the model in order to optimize it for our engine. The most difficult part was probably making a build compatible with the glasses. First, we created a mobile app that worked fine on phones, but not on the glasses. It took a long time to work out the bugs in order to make it work.”
The app lets users not only to learn new things about modern Italian sculpture, but also to solidify that newfound knowledge in their memory. A special minigame is included in the app that asks the player to decide which other pieces were created by the same sculptor.
The team that finished second also developed a virtual tour guide that would educate visitors on the interiors of rooms that once housed imperial ministries.
As the developers explained in their presentation, their end goal would be to have the guide transform itself depending on the location: for instance, it would don an authentic historical costume during a visit to a former civil servant’s office.
The winning team, however, decided to do away completely with the idea of a digital guide. Instead, they propose to enhance the statues with animated backdrops of Italian landscapes.
“Our project is titled “Sculpture Garden”,” says Ekaterina Svyatlovich, a member of the team and a graduate of ITMO University’s Master’s program in Lighting Design. “Initially, the specialists who restored the General Staff Building wanted to turn the hall which currently houses Italian sculptures into an actual living garden. That was one of the threads we latched on to. We also noticed that the sculptures themselves are placed chaotically as if they were trees. We wanted to transport our viewers into an unusual, not museum-like environment, one that’s light, natural, and organic but still conducive to contemplating the sculptures. We wanted to erase the line between museum and the outside, to create a new world within the Hermitage.”
In the end, for viewers it is as if the sculptures are returned to the places where their creators had once drew inspiration for their work from.
“Our biggest challenge was the technical side of it,” says Leisian Mustaeva, another member of the team. “We went to the Hermitage to do photogrammetry of the sculptures so that we could later insert them into our dynamic content. At the same time, it was extremely important for us that the art pieces always remained the core focus for the viewer.”
About AR-CRAFT
The competition launched on April 15, 2019 and brought together students from six higher education institutions of St. Petersburg. The participants were tasked with developing an app for State Hermitage Museum visitors that would be compatible with the Epson Moverio BT-350 augmented reality glasses.
The organizational team’s intent was to bring attention to the use of new technologies in museum environments. The participants also received prizes from Epson and Ascreen.
“For the participants, this is a source of experience and a mark on their portfolio,” sums up Artem Smolin. “For the Hermitage, this was a showcase of the potential of AR headsets. Not smartphones, which are now a common sight in museums, but glasses. In the future, it would be great to see visitors explore museums wearing augmented reality headsets that provide them with fascinating and interactive content.”
