Specialists From ITMO Develop Unique AI Talent-Scouting Service

A physicist, mathematician, psychologist, and engineer all in one.

Interdisciplinary is trendy: research expands its scope, technologies become more complex, and projects and competencies intersect with increasing frequency. High-tech markets show a demand for qualified staff with expertise in multiple subject areas. But the issue is that there are few such specialists: most are already employed, some are abroad, and others simply reside in remote regions and aren’t willing to move. Existing job-hunting services aren’t always an option – sometimes, these people aren’t even interested in posting their CV online.

Researchers from ITMO University’s National Center for Cognitive Research (NCCR) – established as part of the National Technology Initiative (NTI) – have developed an AI-based service that searches for candidates in open sources using keywords associated with the qualities of interdisciplinary specialists. That includes, above all, Scopus, eLibrary, the Russian Science Foundation’s information and analytics system, and the database of the Federal Institute of Industrial Property.

“This service, which we’d started developing almost a year ago, is designed to find unique specialists. These are people who aren’t looking for jobs. They’re already happy with their employment situation, which means they have no CV and therefore there is minimal information about them available to us. But they still have a distinct digital footprint: their articles are published in journals, they present at scientific conferences and appear in mass media. All this allows us to not only identify unique specialists in the corresponding subject areas, but to also find ways to contact them via affiliated universities listed in these papers,” explains Oleg Basov, the head of the project and the Cognitive Non-Verbality Laboratory at NCCR.

Digital profiling and emotion analysis

Exclusive is designed to search for executors of short-term projects, meaning that the right specialists must be identified quickly, but accurately. After all, there is no time for qualifying periods or test assignments.

That’s why the system was trained to not only find candidates, but to assess their experience, too. Their level of qualification is determined based on their number of patents, articles, citations, co-authors – as well as their h-index.

Then, the algorithm follows the candidate’s digital footprint; using the specialist’s name, contact info, and photo, Exclusive searches for matching descriptions in open sources, including social networks and specialized websites. Thus, the candidate’s “personal record” is filled out with information on their education, achievements, contacts, interests, and even bad habits. As for things that the program can’t nail down through analysis, they are automatically added to the list of questions for the initial interview.

During the online interview, the link to which is sent to candidates along with the initial offers, NCCR’s other technology – a digital profiling algorithm – comes into play. This system is able to assess the level of honesty with which the interviewee responds to questions.

“The algorithm detects various traits from two data streams: the sound stream, meaning the speech, and the video stream of the candidate’s face. An increase or decrease in the volume of their voice relative to the average, changes in tone, hoarseness, number of words per minute, pauses, and even facial expressions and gestures– all these things are analyzed in order to determine the truthfulness of their answers,” explains Mikhail Zheludev, a junior researcher at NCCR.

The data collected from speech is combined with the analysis of the video stream; the neural network is capable of analyzing facial expressions (through movement of eyes, mouth, nose, eyebrows, and facial muscles) and identifying emotions. The combination of these two streams, note the developers, has allowed them to achieve an 83% accuracy in predicting whether the speaker is lying or telling the truth.

Ranking the candidates

Once all the information has been collected and assessed, a personal profile is compiled for each candidate. This profile includes their personal and professional qualities and an assessment of their reliability. According to Olga Gofman, a lecturer at ITMO’s Institute of International Development and Partnership who is responsible for the project’s moral and psychological aspects, “reliability” in this case refers to the person’s readiness to follow rules, regulations, and instructions.

“The factors of unreliability include various types of dependency – for instance, on drugs, alcohol, or gambling. These factors can be further exacerbated by poor living conditions, major debts or overspending, affiliation with minor fringe groups and sects, compromising connections, psychological disorders, criminal history, and so on. In different situations, these factors may have different levels of importance. For instance, there are occupations in which history of legal conviction is unacceptable – like teachers and lecturers. In the case of research, for example, we’re unlikely to hire someone with alcohol dependency,” she comments.

Naturally, reliability assessments can be highly subjective and very much dependent on the personality and mood of the interviewer. But in the case of Exclusive, claim the developers, the subjectivity factor is eliminated. Moreover, information comes from various sources: social media data, answers given during the interview, open sources, and the automated profiling system. All this makes it possible to create a highly comprehensive image of the candidate.

Oleg Basov tells us that the technology is already being tested at ITMO University and its partner university RANEPA. It is already a hot topic of discussion in the scientific community.

“We liked the fact that this service rids us of the need to check each candidate’s scientometric indicators on various platforms. It also makes our work easier by automatically collecting various personal information about the specialists,” notes Pavel Golosov, head of RANEPA’s Center for Digital Solutions and AI.

In May, the developers will begin to accept applications from those interested in testing the service for free.

“We believe that AI technologies can help scientists and developers find each other and create professional, capable teams,” he concludes.