Jack and Jill: Researchers Develop Novel Test to Efficiently Measure Human Cognitive Abilities

Why is working memory important? 

Working memory is a temporary storage system necessary for successful learning. This is a primary factor that affects our cognitive skills and even predicts such abilities as reading comprehension, subject proficiency, and academic performance. 

It consists of several components, one of which are visual-spatial skills, which are responsible for our achievements in various fields, from learning languages to mastering scientific fields. Working memory skills can be accessed using a wide range of tasks, including the so-called Mister X task. In this task, participants are shown a picture with two characters holding balls in their hands. One of the characters is occasionally rotated, thus changing the position of the ball. The task of the participants is to identify whether the two figures have the balls in the same hands, remember their location, and reproduce them later.

How was the test created?

Inspired by the Mr. X test, a group of researchers from Tomsk State University, Goldsmiths, University of London (UK), ITMO University (Russia), the Max Planck Institute for Empirical Aesthetics (Germany), Macquarie University (Australia), and Sirius University (Russia) has developed a new improved method for efficient assessment of visual-spatial working memory. While working on The Jack & Jill Working Memory Task, the scientists turned to computerized adaptive tests (CATs). These are computer-aided tests that adapt to the examinee by offering different questions to match their ability level. The probability of a test-taker selecting the correct answer was calculated using the item response theory (IRT).

The researchers ran four trials with participants of different ages and nationalities to create their testing system, compare it with its analogs, and evaluate the accuracy of its measurements. The first trial covered adults (18-68 years old, mean age – 31.3) in the UK and the US who took the test at home. The participants of the second experiment (18-50 years old, mean age – 26.4) from the UK and Australia performed the test at labs. And the latter two studies involved Russian adolescents (14-17 years old, mean age – 15) who demonstrated significant achievements in natural and engineering sciences, arts, and sports. 

What results have been achieved?

The studies undertaken resulted in a novel open-source and reliable test for assessing visual-spatial skills. The researchers compared their product to other testing systems available on the market: backward digit span or BDS (the task is to repeat numbers in the reverse order), Corsi block-tapping test or CBTT (the task is to reproduce a sequence of blocks), and memory updating figural or MUF (the task is to click on dots appearing at the corners of a rectangle).

“We studied the correlations between the performance in our test and in other established testing systems to examine the accuracy of our measurements. It’s vital to demonstrate that our test possesses convergent validity, that is, its results correlate to those of the globally recognized and adopted systems. We looked into three different measures and studied how they correlated to our systems using various samples. Everything works correctly,” says Maxim Likhanov, a senior researcher at ITMO’s ChemBio Cluster.

The researchers also demonstrated that test results are strongly associated with the examinee’s spatial abilities, non-verbal intelligence, and academic achievements. One of the studies even showed that adolescents who demonstrated better results in sciences had higher scores than their peers. 

As noted by the team, the Jack and Jill task is far from simple since it cognitively challenges the memory of participants who have to memorize the position of a ball, mentally rotate a figure, and visualize the entire sequence when submitting their solution. 

How can one use the Jack and Jill task? 

The developed test can be used to study the role of working memory in mastering mathematics, developing creativity, and overcoming ADHD (attention deficit and hyperactivity disorder). On a more practical note, it can also help identify behavioral problems and ways to enhance academic performance among school students. Further research requires more data, including the study of how people of different ages perform in the test. 

The scientists plan to apply their test in other projects, as well, for instance, to study the predictors of success in chemistry at school. As part of the Chemistry Neuroeducation research, ITMO is about to host a large-scale longitudinal study, which will look into working memory as an example of such predictors along with spatial skills, number sense, and academic anxiety. Meanwhile, researchers at Goldsmiths, University of London already used the test in their study on musical capacity. 

A demo version of the test is available online and can be easily used by any scientist for their projects.

“One of the challenges that scientists have to face today is the replication crisis that fueled the concerns about the credibility of scientific findings. The same is happening in psychology, too. Scientists are doing their best to overcome this crisis, and one of the ways to do so is through open science, which is when all research, data, and tools are freely available online. That’s why we decided to develop an open-source test in Russian, English, and German. Moreover, our article that includes a thorough description of the product is an open access one, as well,” comments Maxim Likhanov.

Reference: Elina Tsigeman, Sebastian Silas, Klaus Frieler, Maxim Likhanov, Rebecca Gelding, Yulia Kovas, Daniel Müllensiefen, The Jack and Jill Adaptive Working Memory Task: Construction, Calibration and Validation (PLOS ONE, 2022).