SMARR (smart manufacturer augmented reality robot) is a mobile robotic platform for facilitating remote presence of humans. The robot’s defining feature is that it functions on a combination of VR and AR technologies. Users can control the system with virtual reality glasses Oculus Rift. The video stream that comes from the fitted-in camera is enhanced with augmented reality, which the computer layers on special markers located in the environment. The system recognizes these markers with the help of synthetic vision sensors. The applied augmented reality consists of tables and other important information.
But SMARR’s potential doesn’t just end there: it is also capable of autonomous navigation. When the robot finds itself in an unfamiliar environment, it goes through different probabilistic methods to create a map of the space. Then it finds its own location on this map.
Thanks to these special features, the robot will come in useful on large-scale manufacturing plants. One of its possible applications is oil-derrick operations: by delegating the task of supervising oil extraction indicators in dangerous conditions to SMARR, oil companies will save money and decrease the toll on staff. Tourism is another field that can majorly benefit from employing the technology in its activities. It will give aspiring globetrotters the chance to explore different cities in the comfort of their home by means of VR glasses.
The project was carried out as part of practice-oriented R&D initiative, a special competition for tech-savvy students held by ITMO. Best student inventions were given financial support and help of a scientific consultant, a role filled in by the university’s leading researchers.
Authors of SMARR and the many difficulties they faced
SMARR is a joint creation of a group of Bachelor’s, Master’s and PhD students from a range of scientific fields: from programming to design and electronics. For each and every one of them, the nifty robot was their first real-life project. According to the project leader Alexey Shchekoldin, the group encountered many unexpected obstacles, but these challenges only inspired them to carry on.
“We faced a great deal of problems we’d never even have thought of,” shares Alexey. “When working on a wheelbase, for example, it took us a lot of time to figure out how the drivers operate because we borrowed our wheels of a gyro scooter. Another time-stealer was the video streaming technology: we spent a whopping half a year on getting the hang of the ZED experimental camera. Layering augmented reality onto the video stream in such a way that the robot wouldn’t give up in the middle of the operation saying that it’s too knackered to continue was very difficult to pull off. We had to go as far as to write our own libraries to make the augmented reality a natural fit and eliminate all the disruptions in imaging. The good thing is that we’ve managed to get a patent out of this painstaking work.”
Evaluation of the project and its first successes
Allowing students to obtain their first practical experience is the main goal of all practice-oriented R&D work. Participants develop unique skills such as how to fill in the paperwork, create technical proposals and negotiate these with the client (who in this case was represented by ITMO University).
The main evaluation criterion for judges was how thought-through a student project was: it shouldn’t have been a mere idea packaged in a PowerPoint presentation but a fully-fledged guideline for developing products or technologies which have a feasible implementation and a clear-cut target problem, says Sergey Kolyubin, Deputy Director of the School of Computer Technologies and Control and scientific consultant of the winning team.
“From the very beginning, the students had to be mindful of business aspects of their venture, as well as potential market competition, costs, labor intensity and consumer appeal. The project hasn’t become a startup yet, but with some additional development, I believe that it can be successfully commercialized,” notes Sergey.
As a scientific consultant, Sergey Kolyubin also helped the team not to lose sight of important technical problems.
“The integration of such technologically complex solutions into a single system can eat up a lot of time or even fail if the authors don’t factor in all the nuances. I’d mark the team’s efforts of implementing their project with a B. They did accomplish a lot, for example, in creating a fully operational prototype, but there is still room for perfection; I’d recommend further improvements in hardware-software integration for the robot to work as accurately as a Swiss clock. I hope that the students continue working on this very promising project beyond what has already been achieved as part of this practice-oriented R&D initiative.”
The team hasn’t launched their invention on the market yet, but has already presented its first prototype at the MMAR international conference and a range of ITMO University's events.
