To be fully autonomous, a drone has not only to analyze the surrounding objects and environment, but also to make its own decisions based on that data. The new autonomous CV system to be created within the project at ITMO will become a universal tool for all kinds of tasks – from detecting crop diseases to monitoring moving objects. It will be compatible with existing Russian-made platforms and those that will be developed down the line. Additionally, the system can be adapted to any architecture if provided with inference frameworks or an SDK (software development kit).
168 million rubles were issued by the NTI Foundation to sponsor the development of the new system. As part of a consortium, ITMO’s National Center for Cognitive Research (NCCR) will be collaborating with the State Research Institute of Aviation Systems (GosNIIAS) and KT-Bespilotnye Sistemy. Researchers from GosNIIAS will be developing neural network algorithms while experts from KT-Bespilotnye Sistemy will be collecting data, as well as testing and commercializing the system. ITMO scientists will be responsible for automating the entire lifecycle of CV model creation and making it accessible to regular users – this will include every stage from data preparation to simulator testing and uploading of AI-based models to drones.
“We will create a machine learning platform that, firstly, will include a set of libraries, and secondly, a set of tools that will allow us to build completely different CV models and decision support systems based on these libraries. It will be possible to use open libraries for civilian tasks such as pipeline monitoring, environmental control, and agrochemical treatment of fields. On the contrary, only trusted AI models will be employed for critical infrastructure. We also plan to incorporate ITMO’s best open-source solutions into the platform,” shares Prof. Alexander Boukhanovsky, the head of ITMO’s National Center for Cognitive Research.
Under the hood of the new system will be a number of AI platforms and frameworks that were earlier developed and tested by the consortium’s members. For instance, the NCCR is the only NTI Competence Center specializing in developing cutting-edge software for sector-specific AI systems development and deployment. On the new platform, the NTI-sponsored SMILE visual programming environment will be used to manage workflow and build models. With SMILE, users can design and train AI systems without any specialized tech or IT knowledge.
Alexander Boukhanovsky. Photo by ITMO.NEWS
“SMILE is a visual interface where users can connect building blocks to create an AI model. The program ‘translates’ these blocks into code performed on computational servers. When the number of building blocks involved becomes too big, confusing the user, they can turn to FEDOT, our framework that can automatically create an optimally balanced part of an AI model in terms of quality and performance. FEDOT also uses building blocks, while also imitating biological evolution: mixing all kinds of models until the best one ‘survives,’” explains Prof. Boukhanovsky.
With SMILE, it will be possible to incorporate various support mechanisms for CV model developers, including testing process automation, model robustness assessment, and explanations for composite model structure, via the integrated large language model.
GosNIIAS is one of the leading developers in the field of CV, robotics, and AI in the military-industrial complex. An integral part of the developed AI solution will be Platforma-GNS, a system for neural network development created by GosNIIAS with support from the Ministry of Industry and Trade of the Russian Federation. The platform supports the full development and application cycle of neural network modules for various products: from data collection and marking and architecture selection to automated testing and importing of trained or retrained neural networks to onboard computers. Platforma-GNS is an integrated environment for working with data and neural networks that includes the domestic deep learning library Plat and a number of hardware implementation tools for neural networks compatible with various platforms, including Russian-developed ones. Today, Platforma-GNS 2.0 is already installed and in use in various organizations and fields, including industry, science, education, and defense. It is currently being implemented at Russian universities.
“The team of GosNIIAS has been developing a trusted machine learning system for the military-industrial complex and aviation since 2018. Such a platform will find its demand not only among developers, but also users; we know that AI can continue training and become more efficient when used. We have already successfully collaborated with KT-Bespilotnye Sistemy in the field of intelligent solutions for drones. We are certain that with new approaches and solutions from ITMO, we will be able to develop a trusted and efficient product based on a compilation of our solutions, data, and libraries,” shares Yury Vizilter, a professor of the Russian Academy of Sciences, the head of the AI and Computer Vision Department at GosNIIAS.
The new solution developed by the consortium will ensure the technological independence and global competitiveness of domestic drones. A prototype is scheduled for release in August 2025.
