We’ve recently learned that VK is updating its "smart news feed" algorithm. You were among those who created the “smart feed” back in 2016. What has changed in these two years?
Our team has grown very strong since then. Back in 2016 we were only a few people and now we have a whole team devoted to machine learning algorithms. We’ve learned what every type of user needs.
One of the key factors now is understanding how everything should work. We are constantly working on updates. After all, even if a problem is solved, there is always room for improvement. In addition, over the past two years, new technologies have emerged that allow us to work on more complex tasks. We know that the news feed is very important for both users and creators, therefore we tackle problems in this area every week, honing it to perfection.
Since 2016, we have added new parameters that determine the content in the feed. We started with a fairly simple algorithm that still solved quite a lot of problems, but now the number of factors has become tens of times bigger, and the algorithms predicting the users’ interests – even more efficient.
However, every ecosystem, in our case - a content ecosystem, has its downsides. For instance, authors try to outwit the algorithms. I am talking about getting “fast” likes, etc. We battle this problem regularly so that the users’ feeds show what is truly important.
What parameters do you use?
There are actually a lot of them. We are paying more attention to the users’ locations, because it is obvious that residents of St. Petersburg are more interested in local news. Obviously, we take into account which platform a user comes from. This factor strongly influences content consumption. On a mobile phone we mostly prefer fast content: short videos, breaking news. When we log in from a PC or a laptop, we have time to watch a longer video or actively immerse ourselves in information.
We also consider the time that the users spend reading the news. Why do we need that? For example, there is a popular and rather simplistic assumption that news with the biggest number of "likes" is in fact very important. However, user behaviour is much more complicated. Often enough we do not like those things that interest us: we read the headline, look at the illustration, but do not like this post. We’ve take all of this into account now.
This year Facebook changed the way its news feed is formed, and will now show posts from friends and family instead of “most relevant content”, which you have called wrong and limiting. Why?
We have a different standpoint on this issue; we stand for balance. There are many equal participants in the content ecosystem. On the one hand, our goal as a platform is to make our users feel as comfortable as possible. Of course, VK is historically a place where we communicate with friends.
On the other hand, every creator is also our user; moreover, they just make our website interesting. We have many small talented authors who, for example, participate in the VK Talents programme, and there is a huge amount of popular media outlets on VK.
Thus, our goal is to create a balance, to understand what is important at what point, and personalize this content. If someone likes to read news in general, so there will be more of that in the feed. Another example: some users read the news in the morning and communicate with friends in the evening. Therefore, our algorithms take into account the time of day when a user interacts with the site. Gradually we add various factors to make the algorithms even more accurate.
By the way, this year ITMO University’s public page also managed to get the so-called “Fire of Prometheus”, which denotes pages that create original new content. The VK Talents program began to operate last year; what results have been achieved to date?
To date, more than ten thousand participants have received the “Fire of Prometheus”. We already have a large community of authors.
A vivid discovery was, for example, the singer Grechka, who is very popular now and even appeared on Channel One. Our algorithms found her and gave her the “Fire of Prometheus” back in January if I’m not mistaken. Thus, our idea was proved: we believed that we could analyze everything that happened, take into account the users’ interests and notice truly talented people who are able to attract attention to their content. At an early stage, our algorithms notice the authors, and then they become even more famous and finally are acknowledged far beyond "VK".
This is but one positive aspect that we see: the users are interested in the talents we find. Thanks to the fact that we communicate with our authors, we have a lot of ideas on how to develop further. We are launching a lot of interesting things this year.
By the way, if you want to know more about Prometheus you should subscribe to the VK Authors community. If you are interested in technology, we have a VK Research community where we present our findings in the field of machine learning. We also have VK Tech, where we talk about the latest technologies in the programming world, and, of course, the VK Team, which announces all the major updates.
One of the requirements for communities that want to get a “Fire” is original content or its interesting presentation. How do algorithms today help to find the original source of the content and show it to users?
We analyze the text, as well as all the images. We can compare two texts and the algorithms will show their striking similarity. We can also determine if an image has a certain primary source, which was created earlier on our platform, even if the image was slightly changed.
We know that every author wants us to protect what they create. Therefore, of course, these algorithms will constantly evolve.
What tasks cannot be solved with the help of artificial intelligence and machine learning? And what do you personally expect from technology?
Driverless cars seem to be a very cool opportunity. The second thing, which seems very interesting and promising, is dialogue systems or smart bots.
Nowadays we see that many tasks can be solved quickly and effectively with the help of messages. For instance, we used to frequently make arrangements by phone. In the near future, I think, we will do it through messengers, and many of our routine tasks will be carried out by bots.
At VK we have big plans for making recommendation service even more convenient. And in this case, the fact that we ourselves actively use what we create comes in handy. Logging on to VK every day, on the one hand, we are happy with what we have, on the other - we understand how to make it even more convenient and therefore we are constantly evolving.
When you graduated from university eight years ago, machine learning and big data was less popular than now. What brought you into this area?
While I was preparing the lecture, I was thinking that university gave me great knowledge, important fundamental principles and taught me to explore almost any subject, to be able to understand it. But in fact no one - neither I, nor experts, nor teachers at the university - can predict with absolute accuracy what will be in demand in five years. Therefore, it is important to remain very dynamic and flexible, not to be afraid to experiment in your field of study.
My first job after graduation (Andrei Zakonov is a graduate of ITMO’s Computer Technology Department – Ed.) was a Java developer. Back then, I was really into programming, but a couple of years later new technologies appeared, and the rapid development of machine learning and neural networks began. Since I was staying up to date on all the interesting things happening in this area, I thought I would try to learn something new. After all, even then it was clear that these technologies can greatly affect what is happening around us. Gradually, I became very fond of it and today I understand that this is one of the most promising areas.
Lots of seventh and eighth graders became winners and medalists at ITMO VK Olympiad this year. Can they engage in machine learning tasks at least at a basic level today?
I think, yes, quite so. On the one hand, it can be extremely complicated. For example, to solve many problems, you need to deeply immerse yourself in technology and develop new algorithms. This is exactly what the VK team does, because we solve unsolved problems.
At the same time it is very cool that technologies are becoming more accessible each year. Already existing neural network and machine learning technologies can easily be applied to a new field. There are many different examples of how you can teach neural networks to distinguish photos of cats from the ones of dogs, determine a person's gender and age, or recognize the subject of a text.
This sort of thing can be done at some basic level; it is not necessary to immerse yourself heavily in this area, you can use ready-made code samples, libraries, frameworks. I advise everyone to look at the tasks in the area they are engaged in, find out what looks like routine enough that a computer can do it for them, and try to solve this problem. If a task looks rather monotonous and not very creative, then, most likely, an AI can already handle it, or will do so soon.
What skills does one need to become a part of the VK team and engage in these tasks alongside you?
In fact, we are actively looking for very experienced people - experts in their field, but, also we cooperate with universities. Our internship program allows someone less experienced to continue to study, while solving our problems. Obviously, solid fundamental knowledge is expected: mathematics and physics help to grasp many things in general.
But, as I said, since we are constantly learning, current knowledge is not always important. You have to welcome new things, be courageous, in a way - to not be afraid of any task, try seemingly impossible things. Finally, you need to be able to solve creative problems. This is important for us, because we are trying to create new ways of communication.
Translated by Pavel Vorobyev
