ITMO Staff Develop AR-Solution for WorldSkills Kazan 2019 Opening Ceremony

On August 22-27, the city of Kazan hosted the 45th competition for young professionals, which brought together 1,300 participants from 63 countries. In addition to musical performances and lighting effects, the grandiose opening ceremony involved unique AR effects developed by specialists from ITMO University. Read on to learn who – and how – created these augmented-reality features for the show.

The opening ceremony of the 45th WorldSkills Competition. Credit: Sechenov.com

WorldSkills Competition

WorldSkills is an international non-profit organization founded in 1953 to bring attention to vocational professions and create the conditions for the development of high professional standards. Every two years, the organization runs a competition in 56 competencies among the young representatives of various professions. In 2019, the 45th WorldSkills Competition was held in Kazan, Russia.

The opening event’s main sponsor was the company Sechenov.com, which can boast a great deal of experience in organizing world-class public events. In addition to musical performances, those in attendance could observe various augmented reality effects such as a flying whale, floating flower petals, and a bright-blue meteor shower.

The opening ceremony of the 45th WorldSkills Competition. Credit: Sechenov.com
The opening ceremony of the 45th WorldSkills Competition. Credit: Sechenov.com

Teamwork

These AR effects were developed by a large team of professionals. The script was created by the staff of Sechenov.com, while the equipment – hardware, tracking systems, communications – was provided by the company LYM.pro. The software and the effects themselves were developed by the staff of the Interactive Visualization Laboratory at ITMO University’s National Center for Cognitive Technologies.

“The basic premise was quite similar to what we’ve done for the European Games in Minsk. Our task was to create augmented reality effects during the opening ceremony that would be visible on television broadcasts and on audience members’ smartphone screens. Both of the software solutions are our own. The biggest difference was that this time around, we had a lot more creative liberty in terms of graphical content, as we developed it ourselves. I believe that this freedom to create helped us find the proper compromise between the quality of graphics and the technical capacities of broadcasting equipment,” says Andrey Karsakov, head of the Laboratory and staff member of the National Center for Cognitive Technologies.

You can see the full broadcast of the opening ceremony here. The show begins at the 1:02:00 mark.

Staff of the Interactive Visualization Laboratory in Kazan
Staff of the Interactive Visualization Laboratory in Kazan

Challenges and novelties

At WorldSkills, ITMO University staff presented five AR creations – “Perception” (for a performance by confectioner Anna Prokopenia, Europe’s representatives in the WorldSkills Champion Trust), in which an on-screen monument transformed into a bar of chocolate; “Experiment” (for Pranav Mistry, Global Senior VP of Research, Samsung), in which a box with Schrödinger’s cat appeared on the monument; “Perfection” (for Ekaterina Bereziy, CEO of Exoatlet), in which a massive whale, symbolic of nature’s perfection, swam in the sky above the stadium; “Creation” (for Dr. David Hanson and the robot Sophia, Hanson Robotics), in which rose petals flew out of a screen as a VR artist drew their painting; and “Concentration of Monument’s Energy”, a performance that took place when the competition’s competencies were being presented.

According to Andrey Karsakov, the show in Kazan was an opportunity to improve on the experience acquired during the European Games in Minsk; this time around, they honed their skills rather than made a debut. Of course, there were still challenges.

The biggest challenge was to develop the proper UX design for the mobile app. The interface was to be simple and accessible so as to allow thousands of people to use it simultaneously without missing out on the show’s most memorable moments.

The opening ceremony of the 45th WorldSkills Competition. Credit: Sechenov.com
The opening ceremony of the 45th WorldSkills Competition. Credit: Sechenov.com

“Users want results in three clicks: install – launch – watch! You need to take into account that users may not be familiar with the technology, may not hear the instructions given before the show, may not understand how things work. No matter how many times we’d write guides and tips, explain which devices support the app and which do not, or how to use it – there are still people who can’t figure it out. During the three events we’ve taken part in this summer, we have collected a great amount of feedback from the users and are already developing a framework for the development of similar apps,” shares with us Andrey Karsakov.

The staff of the Interactive Visualization Laboratory at ITMO University’s National Center for Cognitive Technologies used the show in Kazan to conduct the first test of its computer vision-based lens and camera angle calibration software. Though this was an experiment and not all of the systems worked as the specialists intended them to, this was a step on the path to complete automation of such processes.

Staff of the Interactive Visualization Laboratory in Kazan
Staff of the Interactive Visualization Laboratory in Kazan

Another of the project’s achievements was the significant decrease in time required to create lens files, i.e. the calibration parameters that describe the optomechanical properties of a lens (FoV, distortions, lens breathing, zoom axis shift).

According to Andrey Karsakov, the team’s future development plans are expansive: after all, interest in augmented reality tech only increases, and there is no software product on the Russian market that would allow broadcasters to use augmented reality and virtual studios in television work. The Laboratory’s future research will be headed in this very direction.

Journalist
Translator
Archive by year: