ITMO Youth Lab’s New Software Prepares Medical Data for Use With AI

Hospitals and outpatient clinics rarely use the same format to create and store patients’ medical data, including test results, check-up records, handwritten notes, and discharge summaries from different institutions. To be used in AI training, such as when developing tools to help doctors make diagnoses or provide medical forecasts, this data needs to be extensively re-formatted. Often, developers have to collect such data manually. A similar issue exists in clinical practice: clinicians often need to review dozens of documents from various sources before reconstructing a complete picture of a patient’s condition. Automating the collection of medical data from different sources can help solve these routine processes.

“The main challenge when working with medical data is its variability and fragmentation. It’s not about processing one specific document, but about dealing with many very different sources. When a developer works with a large amount of clinical data, 60-70% of time needed to train the model is actually spent on data preparation. Documents need to be rid of personal data and technical notes and brought to a single format, while key medical information – diagnoses, symptoms, prescriptions, and test results – has to be properly marked. MAS-EHR can simplify and accelerate this process,” says Ivan Zolin, one of the platform’s developers, a student in the Deep Learning and Generative AI Master’s program at ITMO’s Institute of Applied Computer Science. 

MAS-EHR (Multi-Agent System for Electronic Health Records) consists of independent modules and follows a stepwise data processing pipeline. One module is responsible for optical recognition: it reads text from scanned copies and tables and converts it into a machine-readable format. The second module identifies and masks patients’ personal data. Next, the third one integrates language models, which extract key medical information from the text, such as diagnoses, symptoms, prescriptions, and test results, and link them semantically. At the same stage, terms are standardized according to international systems such as disease classification codes (ICD-10) and SNOMED CT, which is a  unified reference system for diseases and clinical concepts. For the final step, a dedicated verification module based on an LLM (a validator agent) checks the resulting structure against the original document.

If a part of the data is lost or the model has hallucinated something, the system notifies the user about this and submits the fragment for another processing round. This approach makes it possible to change and additionally train separate platform components without disturbing the entire system, as well as to easily integrate the system into existing clinical AI solutions, such as digital medical records.

Here’s an example of how the system works. First, it receives several months’ worth of a patient’s test results. MAS-EHR extracts specific indicators, compares them with diagnoses and symptoms from different documents, and connects them into a single system. The result is a complete picture of changes in the patient’s state, what recommendations they received, and how their body reacted. This way of representing data makes it convenient for clinicians to quickly gauge patients’ medical history and for developers – to train medical AI models.

“Major tech companies are developing proprietary cloud medical services for document processing, but their users receive ‘ready-made’ answers and have no way of knowing how the result was produced or if a model made a mistake. MAS-EHR takes a different approach: a clinician or a developer can trace what happens to data at every step. Each process, including text extraction, data anonymization, and how the terms were linked, is verified. At the final stage, the validator compares the original document with the result. Thanks to compact fine-tuned models, the system operates faster and uses fewer resources than large, general-purpose solutions,” adds Ivan Zolin.

The platform was developed at the Digital Public Health Technologies laboratory, where Ivan Zolin collaborated with Vladimir Skvortsov, a student in the AI Product Management Master’s program, Artur Faizrakhmanov, a developer at the lab, and Vyacheslav Chukanov, Ivan’s research supervisor. The lab provided the team access to anonymized medical data, as well as expertise from practicing specialists from Fomin Clinic and the N. N. Petrov National Medical Research Center of Oncology. Their feedback helped the team understand how their platform should work in medical practice and what data is most important for medical professionals.

Ivan Zolin presented the platform at the international university awards in AI and big data Gravitation, where the project won in the youth category and caught the attention of several major Moscow clinics. The team is currently in negotiations about the pilot implementation of their solution.

“The main aim of our platform is to bring order to places where there is chaos. We turn fragmented medical documentation into a single structured source of data about a patient’s health aligned with widely accepted healthcare ontologies, nomenclatures, and reference systems, so that this data can be effectively used in research, analytics, and medical AI systems,” says Anna Andreychenko, the head of ITMO’s Digital Public Health Technologies laboratory. 

In the future, the team is planning to improve their project and train the platform to not only create clinical records, but also predict possible progression of diseases.