The issue

A key early stage of any machine learning or predictive modeling project today is a comprehensive analysis of the system in question. Knowing everything about a given nanoparticle allows scientists to not only predict the features of novel materials more accurately but also find ones with certain properties among all the possible options.

There is no systemic approach to describing particles: scientists have to go through multiple papers manually and oftentimes either use average values or even exclude certain parameters when trying to characterize systems. This hampers the use of AI tech in materials science. Machine learning algorithms fail to process data, draw conclusions, and learn because there is close to no data about the surface structure of particles and their shapes are usually described using general terms (e.g., spherical or cubic) that do not say much about the properties of these particles. 

The algorithm

The new method developed by researchers from ITMO solves all these problems. The algorithm analyzes an image taken with an electron microscope and produces data on the characteristics of these particles in the quantitative format. Yet the data obtained is not easily parsed, meaning that it is not the case when the first figure is associated with the shape of the particle, the second – with its size, and so on. This information is summarized and hidden from users up until the parameters of the original image are restored. A similar mechanism can be observed in neural networks that compress images of faces into sets of indicators, each responsible for a particular feature of the face that can’t always be understood by humans. 

“The developed algorithm works with an array of numbers that are extracted from images and describe their features. Based on this data, the program searches for similar particles in its datasets. It can distinguish them morphologically and identifies other parameters, too. For instance, when we tried to find cubic structures in our database using the algorithm, it provided us with examples that suited our request most,” says Nikita Serov, an engineer at ITMO’s ChemBio Cluster. 

Nikita Serov. Photo by Margarita Erukova / ITMO.NEWS

Nikita Serov. Photo by Margarita Erukova / ITMO.NEWS

During the first phase of the study, the researchers developed a database of nanomaterials using a collection of the texture characteristics of calcium carbonate. The substance was chosen deliberately. The reasons behind this choice is that the synthesis of calcium carbonate (i.e. the required concentration, temperature, storage period, as well as obtained quantities) can be easily described and the material greatly varies in shapes and sizes. Calcium carbonate is a common natural substance: many living beings use this mineral to form their shells and so on. The modification of its synthesis process can result in a rich set of the most elaborate morphologies. 

The method was also tested on gold nanoparticles, which are especially common in photonics and drug delivery because of these fields’ focus on structure. The scientists used data from various scientific articles for testing. The experiment demonstrated the efficacy of the algorithm on gold nanoparticles, as well. 

Not only images

Researchers can also use the algorithm to create materials in reverse order. All they need to do is to submit the required properties and the algorithm will show how to produce this type of material. This can be done with images, too. If a scientist wants to receive a certain morphology, they can draw a shape they need in a photo editor and add desired surface structures, such as squiggles or spines, from the original particle. The proportions between such elements must correspond with the real material. The database automatically converts drawings into contour images, thus making the search possible. 

“Our algorithm works just like Google’s reverse image search feature. In less than a minute, it finds a similar nanomaterial based on the user’s inquiry and suggests ways to synthesize it. Even though the program already demonstrates great results, we plan to continue optimizing it so that it could work even more efficiently with bigger datasets,” adds Nikita Serov. 


The researchers believe that the new method will allow them to automatically obtain data about particles in order to study them, compare them to other substances, and develop new compounds. The surface structure of the particle, for instance, affects its behavior in living organisms: its shape influences cellular immunity and metabolism, and its structure – optical properties. All these insights could be beneficial for biomedicine, photonics, and many other fields. 

The team is still working on predictive models and testing their accuracy. At the moment, the database and algorithm are available as code on GitHub. As stressed by Nikita Serov, the team strived to make their algorithm as user-friendly as possible. In the future, the researchers will turn their technology into a full-fledged website.

Reference: Inverse Material Search and Synthesis Verification by Hand Drawings Via Transfer Learning and Contour Detection (Small Methods, 2022) by Nikita Serov and Vladimir Vinogradov.