What is evidence-based education

Just like evidence-based medicine that favors experimentally proven treatments, evidence-based education is all about relying on research when developing educational programs or new teaching methods. Dmitry Abbakumov believes that this approach to education ensures that teachers use the techniques and tools that will help their students achieve the desired results. According to research described in Teaching Today. A practical guide by Geoff Petty and Visible learning by John Hattie, the evidence-based approach compared to the more conventional one is correlated with higher marks and better learning aptitude. 

Even though it might seem that evidence-based learning is a recent invention, its foundation was actually laid over 100 years ago by American psychologist Louis Thurstone, who came up with the learning curve equation. Coined in 1919, the learning curve describes how a person’s skill in a certain task changes through time or depending on their experience. This concept is still in use today, for instance, in adaptive learning systems, where educational content is hand-picked for each individual student.

In Russia, this field was first introduced by psychologist Aleksandr Nechaev, who founded experimental pedagogy in the 1910s-1930s. He was interested in the way school students with various aptitudes learn, the results they achieve, and ways they can be taught efficiently. 

There are three key components to evidence-based education: quality data, valid statistical models (with values as close to real life as possible), and transparency of suggested data and models for teachers and students.

Challenges as a means towards an efficient academic system

In the course of learning, all of us have to face difficulties, which can be random or systematic. The former are connected more to each individual case and can be remedied by feedback to students, but in case of systematic challenges, we need to consider changing the entire approach to education. 

One of the most common methods to deal with systematic challenges these days relies on reviews of the educational course or program. For instance, if methodologists notice that students keep making mistakes when solving a particular task, the answer might be to make the task simpler or add notes, hints, and examples. However, this method only works after the challenges have snowballed to the point when they can’t be ignored, whereas a more efficient way would be to identify potential problems in the early stages of implementing a course or a program.

What happens when we consider course contents as linked elements and hypothesize that students face difficulties with one task because they didn’t get those that they studied previously? In order to answer this question, psychologist Sacha Epskamp came up with the psychometric network that helps researchers analyze the connections between processes in academic and psychological systems. For instance, it can identify how challenges with one task can be linked to mistakes in others, as well as highlight the similarities between these tasks. Thus, this approach introduces the concept of “connectedness.” 

an introductory lecture to network psychometrics by Sacha Epskamp

In psychometrics, two types of problems are called “connected”: the ones testing a key skill and the ones testing a basic concept. The key skill tasks are rather challenging for most students, and mistakes in these tasks cause mistakes in others. Conversely, the basic concept tasks are easy for the majority of students, but those who do make mistakes in them will likely have difficulties completing other tasks as well. These two types of tasks are the fundamentals of learning, and by paying attention to them, we can prevent further difficulties in the educational process.

Experts at Yandex are currently working on a new tool for online programming simulators based on Epskamp’s model. This way, they can analyze a combination of attempts by a single student, as well as the amount of time a single solution took them, the efficiency of this solution, and more. Using the model, the experts are planning to improve the educational content, the feedback and support provided to students, as well as recommendation systems for revision.

ITMO’s take on Data-driven approach to education

At ITMO learning analytics specialists collect data used as a basis for various management decisions in what is referred to as data-driven approach to education: relying on factual data and analytics in managing educational programs and evaluating the efficiency and statistical validity of the involved decisions.

Svyatoslav Oreshin, the head of ITMO’s Learning Analytics Center, says: 

“Our decision-making algorithm works as follows: let’s say we receive a request about students getting expelled from a certain program. First, we will have to confirm or reject this hypothesis using the data we have, because if we don’t do it, it will be akin to making our decisions blindfolded. Our main goal is maintaining our focus on the initial problem and ITMO’s global objectives, taking facts into account. In order to achieve this, we need to establish a system of metrics and OKR (objectives and key results) related to our task – if we analyze every bit of data we come across, we can acquire nearly any results. This is why we need to maintain objectivity and not only identify factors of a problem, but also evaluate the existing solutions and their contribution into the university’s goals. Otherwise, as the saying goes, “without data you're just another person with an opinion.”

Among the center's projects is ITMO.LENS that provides computer vision solutions for smart campus and implementation of the data-driven university concept. Another initiative that stands out is ITMO.TRACK that gives every student the opportunity to build their individual learning tracks by combining the fundamental approach of university education and the edtech approach used by commercial companies (Skillbox, Yandex Practicum, etc.).

“ITMO.TRACK consists of a number of machine learning models and AI algorithms, which help us translate student’s interests and career aspirations into the language of courses and educational programs. This way, we can build learning tracks based on maximum compatibility between the courses and skills desired by a student. Apart from developing the associated algorithms and user interface, we systematized the data about the academic process and labelled the available courses,” adds Svyatoslav Oreshin. 

At ITMO, data is collected on various levels, from the entire university to a faculty, a program, a course, or even a single class. Among all collected information, there is subjective data acquired through surveys of graduating students – and its subjectivity and non-reproducibility are its main disadvantages. That’s why experts at the Learning Analytics Center choose to rely on more objective data acquired through the university’s information system ISU that contains details about students, lecturers, transfers, readmissions, academic leave, expulsion, and other procedures. Thanks to this data, the specialists can build digital representations of academic process participants, which are essentially dynamic vectors described by hundreds of different features. This method facilitates hypotheses testing and the process of training and implementing machine learning models.