Five years of MOOC
The term MOOC (massive open online course) was coined in 2011 by professors from the University of Stanford. In five years, the cash flow total for MOOC-provider companies has reached 400 million dollars and more than a thousand people were employed in that industry. Each year saw growth in the number of courses, too – just two years ago, 1800 new courses opened, as compared to 2016’s 2500. According to data by Class Central, users now have access to approximately 7000 MOOCs from 700 universities all over the globe. Even the famously conservative Oxford University has recently began a MOOC program. Last year, MOOCs saw 23 million first-time users. Thus, the total number of people who have signed up to at least once course is 58 million.
But the low outcome numbers of MOOC remain a burning issue in online education. Earlier on, analysts from MIT and Harvard published the results of a joint study that concludes that only 5% of students who sign up for MOOCs actually complete the courses. The experts analyzed 17 courses based on the edX online platform and the records of more than 800 thousand “attendees” of those courses. Still, despite the low number of graduates, two thirds of the students have “gained some experience” in the process, notes one of the study’s authors, MIT professor Isaac Chuang.
Reasons for poor performance
According to Lyubov Lisitsyna, head of ITMO University’s Department of Computer Educational Technologies, there are at least three basic factors that determine the efficiency of a MOOC.
The first is the students’ potential lack of preparedness for online learning. In other words, many students can’t handle such a study format (can’t work with rapidly changing UI, react in timely manner to change in assignments, graphs, tips and other course elements). Because of this, those who apply to a MOOC need to already be apt at working with various electronic forms and to have well-developed cognitive functions – attentiveness, mental processing and memory capacity.
The second factor is the emergence of issues among students during the completion of interactive assignments that employ various forms of informational technology. And, finally, the third factor that complicates the work of today’s online courses is the decrease in users’ interest in subject matter and their lack of motivation to continue their studies and to complete the increasingly difficult assignments.
The solution
In their paper, Ms. Lisitsyna and her colleague Evgeny Efimchik have offered their solutions to these issues and reinforced them with the data gathered during development and deployment of ITMO University’s MOOC course on the “Open Education” platform, which currently hosts 22 courses developed by ITMO.
To counter the first issue – the users’ lack of preparedness for online education – the authors propose a special technique that aims to increase the students’ cognitive abilities.
“It is no secret that user interface in online education services contain a great deal of information that is constantly being updated; there are all sorts of graphs and information fields. Someone who hasn’t had any experience with online education would have a hard time handling this amount of information – they simply lack the cognitive ability. Which is why we propose to preface online education with situational awareness training that will, on the one hand, increase the users’ basic cognitive functions; primarily their attentiveness and rate of response and, on the other, have a positive effect on the educational performance,” – says Ms. Lisitsyna.
The "Methods and Algorithms of Graph Theory" course on the Open Education platform.
The training programs are developed based on a series of psychological continuous performance tests developed by American researchers (Continuous Performance Test, CPT и Psychology Experiment Building Language (PEBL) Test Battery, Version 0.14) and are accessible in a software form. The program includes several types of tasks that aim to train a user’s attention, memory and rate of response.
The development has already been made a part of ITMO University’s online education system AcademicNT and used for situation awareness trainings for first- and second-year university students. As Lyubov Lisitsyna notes, the program has already proven its effectiveness in practice.
The system was first tested by students studying computer science and discrete math. The experiment was conducted as follows: the students had completed an online exam on a particular subject and were then separated into a reference group and an active group. For a month, the active group’s members would attend situational awareness trainings while the reference group was left uninvolved. For the duration of the experiment, both groups did not attend any classes on the subject of the exam. After a month, the students completed the online exam for the second time. The results of the test showed a 14% increase in average scores among the members of the active group.
“The experiment showed not only a significant effect of such training on the basic cognitive functions of the test subjects, but that it is transferred into the results of their online education. The effects of such training fade with time. For that reason, such trainings can be offered to users every time that their efficiency appears to drop. In addition, those who apply for online courses for the first time may be offered such training to prepare them for the online education experience,” – explains Ms. Lisitsyna.
But how does one solve the issue of users failing to cope with the increasing difficulty of their course assignments? According to the researchers, the cause of this issue is the variety of types of user interfaces used by MOOC developers. Every time a user is given a new type of an assignment, they have to spend a considerable amount of the time learning how to navigate through and fill out the forms of the unfamiliar new interface. This results in users making less effort to complete the assignment in question. Meanwhile, the time allotted for the completion of the assignment remains the same.
Open Education platform.
This issue could be solved by improving a user’s situational awareness through the creation of a more uniform user interface in the confines of not just one online course, but the entire online education platform. As of now, approximately half of ITMO University’s courses on the “Open Education” platform use RLCP-compatible virtual laboratory technology to present their course assignments. A common look was developed for the various interface forms used in the courses to facilitate the user experience and to increase the courses’ effectiveness.
To test this solution, the authors analyzed the amount of time used by students to complete their assignments in the “Methods and Algorithms of Graph Theory” course. The experiment used the data of 152 students who completed 11 of the course’s assignments.
“When we analyzed the data it became clear that the introduction of uniform templates shortens the amount of time between the first attempt and the completion of the assignment – all that despite the increase in difficulty. This shows that users spend most of the time on the contents of their assignment and not on learning the interface. The use of templates allows course developers to avoid having to “invent the bicycle” every single time and to present their tasks in a form that is more comfortable to the students,” – explains the researcher.
Online course platforms often publish leaderboards that display the names of the users who are the most successful in their study. But perhaps this simple game mechanic could be improved by offering the students the ability to compare not only the results of their regular tasks, but to try their hand at tasks with increased difficulty? The perfectionism mechanic is intended to make use of that concept and to provide additional motivation to students who are intent on attaining the best results. So how does that work?
“Perfectionism is a mechanic that entails showing the users that not only have they completed their assignment, but that they can also try their hand at a tougher version of the assignment. And we did see many of the users respond well to that mechanic. When we tested it in our web development course, we discovered that it increased the course’s effectiveness by 13%,” – says Ms. Lisitsyna.
Lyubov Lisitsyna.
Therefore the students who respond to such mechanics are not only motivated to complete a course’s basic requirements, but to keep going and to reach even better results. In addition, the perfectionism mechanic is much more effective than the rating mechanic as it ensures a better involvement in the educational process by the more engaged students.
International recognition
The results of the study conducted by ITMO’s researchers were presented at the international conference on Smart Education and e-Learning (SEEL-2017) that took place on June 21 to 23 in Vilamoura, Portugal. The SEEL conference is a part of the Smart Digital Futures 2017 multi-theme conference that brings together researchers from USA, Europe, Asia and Australia. The annual events are organized by Bournemouth University (UK) and Bradley University (USA).
The ITMO researchers’ paper was one of the four nominees for the title of the Best Conference Paper of SEEL-2017. The jury consisted of 18 expert researchers from US, Australia, France, UK, Romania, Poland and others. They noted the paper’s relevance and the effectiveness of the methods described therein. A total of seven research articles by ITMO’s scientists were presented as part of SDF-2017.
“Our paper contains an analysis of the factors that affect the rate of effectiveness of MOOC and our proposals for possible solutions to that issue, as well as experimental data. We would, however, like to note that the factors analyzed in our study are just a part of a bigger list. Our paper opens a new page for the conference. Hopefully, by next year (The 5th SEEL conference will be held in Australia in June 2018) more scientists from other universities will, too, turn their attention to this issue and we will be able to see the big picture. After all, this was a topic of discussion at the conference’s final closing session,” – concludes Ms. Lisitsyna.
