ITMO Scientists Build Compact Neural Network for Biomolecules

In medicine, peptides can be used as a targeted treatment to deliver drugs to cancer cells without damaging healthy ones. Some of these compounds can also serve as a promising material for creating therapeutic agents due to their antimicrobial and anti-inflammatory properties.

Their biological activity depends on a set of amino acids and their arrangement within a chain. To evaluate the therapeutic potential of a peptide, researchers must first transform the data into a format that is suitable for machine learning – for example, a set of numbers that describe the molecule under study.

For this purpose, scientists turn to large-scale protein language models (pLMs) – specialized neural networks trained on a massive amount of data that represent amino acid sequences as text and can predict the properties of proteins and peptides. Such models generate highly accurate predictions across several key peptide-specific metrics – from an antimicrobial activity to solubility. However, this approach has serious limitations. First, the model contains billions of parameters and therefore demands powerful computing resources to operate. And second, AI is seen as a black box: its internal workings remain a mystery to researchers and can only be “deciphered” via additional analysis methods, which are oftentimes misleading. 

The ITMO team decided to take a different approach and created a compact autoencoder called dcBiLSTM‑AE that has peptide representations initially tied to their specific physicochemical properties and require no additional training. Therefore, the model not only delivers the result – but can also explain what peprtide property affected it.

In the new model, a peptide is described not as a text – but as a table of physicochemical properties. The rows of the table represent 46 peptide descriptors (charge, hydrophobicity, molecular weight, etc.) and the columns – the position of amino acids in the chain. Unlike those of protein language models, each cell here has a clear physics meaning. The autoencoder then shrinks it into a compact peptide description. To avoid mixing properties, the model was specifically trained to account for their individual contributions.

“Our model has about 90,000 parameters and popular protein language models have millions and billions of those. For pre-training, we used roughly 155,000 peptides, which is far fewer than needed for typical protein language models. Thanks to its compact architecture and use of physicochemical properties, we are able to generate quality peptide representations using a smaller model and less data. And, as our analysis shows, for some practical tasks, these representations are comparable in quality to those of larger models,” explains Evgeniy Nam, the first author of the paper and a PhD student at ITMO’s Advanced Engineering School in Interdisciplinary Engineering.

The model has been tested on eight tasks, seven of which were related to predicting antimicrobial, anti-inflammatory, antidiabetic, antioxidant, and hemolytic activity (the ability to destroy erythrocytes), nonfouling (resistance to protein adsorption), and solubility. The eighth task was a regression one: the model was expected to predict the minimum concentration of a peptide needed to prevent the growth of E. coli under laboratory conditions. Such calculations are used in drug development to evaluate the efficacy of a potential medication and determine the necessary substance dosage.

Overall, the model demonstrated a performance comparable to that of large protein and peptide language models; in two out of the eight tasks (namely, predicting anti-inflammatory activity and minimum inhibitory concentration) it ranked second among six models.

The model generates a compact peptide description, which can be integrated with other algorithms to solve a range of problems – from discovery to fundamental research. In practice, it eliminates the need to go through thousands of candidates for potential medications and in terms of research, it allows scientists to deeper examine the structure and functions of biomolecules and identify correlations between their structures and properties.

In the future, the team plans to scale their approach to larger chains and rare amino acids that are not as widely occurring as the 20 natural compounds, as well as adapt it for ones with unusual shapes – i.e. those with branches or a ring structure. These shapes can enhance a molecule’s bioavailability  or help it stay in the bloodstream longer.

Designed for research teams and drug development companies, the model is not bound to a single function; instead, it is suitable for various tasks and, after retraining on more complex peptides, can be incorporated into company software to accelerate the initial candidate screening and reduce the cost of laboratory testing.

The research was carried out as part of the Priority 2030 program.