The chatbot designed by ITMO University graduate wins the Turing Test Contest

The chatbot called `Sonia Guseva` simulates a 14-year-old girl. Digital Sonya was able to trick 7 experts out of 15who thought that they were chatting with a human. That is why the chatbot came first in the Russian Turing Test Contest 2015. It was in the framework of Startup Village and took place in Skolkovo. This year the winner took the first prize easily because `Sonia` deceived 47% of experts while the chatbots which came second and third deceived 24% and 19% respectively. Ivan Golubev, the designer of the `Sonia`, told us how to program a human-like talker.

Tell us, please, about the contestrules.

It was organized by Nanosemantics, Inc. which has the service for chatbot designing. All participants could use the Knowledge Base, the log page and the page for testing. To control the styles of all texts I disabled standard settings and wrote the hand-code.

Each chatbot has 15 three-minute “talks” with the judges. Then the experts determine who has talked with them, the machine or the human.

The chatbot should be as human-like as possible. That is why I made up the reliable facts like the names of her parents and friends, favorite school subjects and so on.

What is Sonia`s story?

Sonia lives with her granny. She studies in the eighth grade. She is not good at math. Her hobby is dancing. She has the best friend and so forth.

Did you analyze teenager interests to invent these details?

I didn`t read teenage forums, but I used the focus group for the debugging of the chatbot.

Do you really think that a teenager is simpler than an adult to be simulated?

I banked on unpredictability. When the adult says something unexpected, it raises doubts. At the same time this behavior is normal for a child.

What reply made the experts think that the `Sonia` is a machine?

The judges invent tricky questions that the robots can`t answer adequately. `Sonia` is wittedbut sometimes her replies were not convincing.

Are there the standard rules for such contests?

There are no standard rules. In 1950 mathematician, Alan Turing supposed that in 50 years chatting with the machine for 5 minutes, there’s a 30% chance that one will not distinguish a human from a machine. So the base of the contests is the Turing Test but the quantity of the judges, of the robots and the duration of conversations are different. This contest is the most important and significant test in Russian language.

Where can these chatbots be used?

I created `Sonia` especially for this contest. She had to deceive the judges talking to them for several minutes. It wouldn`t be used somewhere else. However, there are robots that are designed for virtual assistance or customer service in shopping malls. Nevertheless, I still think that the robots will not replace people in the nearest future. There is a lot to be done in this field.

Aleksandr Pushkash

News portal editorial office