ITMO Scientists Launch App for Meditation Quality Assessment

Meditation as a practice

Meditation is a commonly used relaxation method that is believed to have a range of positive effects on the human brain. Lately, more and more people have begun to practice meditation techniques using videos and audio guides or specialized apps. Although meditation apps have become increasingly popular, they cannot completely replace experienced coaches who can provide people with much-needed feedback and track their progress. 

One of the possible solutions may be found in computer algorithms: for instance, video analytics systems that can determine the position of the body based on its key points. A research team from ITMO has been working on such algorithms for some time. For example, they had previously presented a system capable of assessing a driver’s physical state and thus benefit road safety. At the moment, the researchers apply the methods developed as part of that project to assess a person’s mental and emotional state during meditation. 

“We’ve proposed a system that detects the user’s body in 3D using its key points. Following optical-flow methods, we examine the difference in the position of these points in each subsequent frame and thus define a number of various parameters: chest movements, inhalation-to-​exhalation ratio, etc.,” explains ‪Alexey Kashevnik, the head of the project, an associate professor at ITMO’s Faculty of Information Technologies and Programming, and a senior research associate at St. Petersburg Federal Research Center of the Russian Academy of Sciences‬.

Assessment

The algorithm uses a phone camera to detect different types of body movements and breathing frequencies, and data from wearable devices – for the pulse and movement patterns (stretching, fidgeting, etc.). The obtained model allows users to receive feedback on their meditation. As a starting point, the scientists chose the most popular type of static meditation inspired by the techniques of mindfulness-based stress reduction. Practitioners must concentrate on their body by, for example, observing their breathing and thereby stopping or at least reducing the flow of their thoughts.

“When you’re meditating, your pulse must be steady and your body – calm. Emotions make your heart beat faster. So, if you’re sitting still but feel anxious, it means that you haven’t achieved a state of inner peace. The same is true when you start to move around, stretch, or fidget,” says Vladislav Malyutin, a member of the research team and a Master’s student at ITMO’s Faculty of Information Technologies and Programming.

Users earn points for staying relaxed and lose them when the app notices sudden changes in their movements or breathing. As a result, they receive a detailed graph and review, as well as practice recommendations. The research has been published in Sensors.

Prospects

The app is now available for free on Google Play, as well as on its official website, where users can upload a video of their meditation and receive feedback. As it implies the use of several neural networks, all calculations are performed server-side.

“The algorithm is based on three neural networks: FasterRCNN for detecting a person in a video, PoseNet for defining key points, and SelFlow for analyzing optical flow. We take two images and calculate displacement vectors for each pixel. By doing this, we can plot a motion graph for each individual key point and a movement heatmap. The color of the heatmap is determined by the direction of the optical flow vectors, and its transparency – by the magnitude of the vectors,” explains Igor Ryabchikov, a PhD student at ITMO’s Faculty of Information Technologies and Programming.

Its database consists of over 70 videos (20 minutes each) featuring 17 people – team members, volunteers, and meditation instructors. The team has recently received agreement from the St. Petersburg-based meditation school Osho Center for creating a focus group and recording and analyzing videos of people during meditation practices.

“We’re currently trying to figure out the value of our invention for society and the economy. Together with Osho Center, we plan to conduct experiments to assess just how promising our technology is,” says Alexey Kashevnik.

As the creators emphasize, the project is designed specifically for beginners and those who are just making their first steps in meditation and want to observe their progress. Meditation teachers can also use this app to track their students’ state and performance.