Challenges
Let’s say you’re watching a YouTube video of the Boston Dynamics robot opening doors. Suddenly, you’re hit with a lipstick commercial. Besides simply being annoying, it also makes you feel like you’re watching TV, where you can’t choose content. But this is YouTube, where you watch what you want to! Of course, it’s impossible to get rid of commercials entirely. But commercials that don’t suit your interests are even worse. If the cute metal “dogs” were interrupted by, say, a commercial for a cool new drone model, you might have even been interested.
“Today’s native advertising is barely analyzed for relevance to content it’s attached to. They offer you diapers when you’re looking for a phone. You search for a bicycle, choose one and buy it – and you’ll be hounded by bicycle ads for months on end. Basically, most context-based advertising only uses demographic criteria – age, gender, etc.” – explains Vitaly Sotnikov.
There’s another issue – no advertiser wants their ads in socially unacceptable videos. This is exactly what happened last November with a number of major companies whose advertisements appeared alongside questionable YouTube videos of scantily clad children under which predatory comments were posted. Companies like Diageo, Mars, Hewlett-Packard, Deutsche Bank and Mondelez had pulled their advertisements off YouTube and Google’s ad service, referring to the situation as “shocking”. In response, Google announced the hiring of 10,000 employees who would evaluate YouTube video-content. The IT giant also announced that it is working on machine learning algorithms to use in screening videos.
NativeOS
The ITMO graduate’s startup aims to remedy these problems. The NativeOS team has built a neural network capable of analyzing not just the objects in the video, but also its emotional and social tone, i.e. its content.
“It’s like in a movie. We understand an episode’s message not through the objects in the frame, but their relationships to each other, the atmosphere, the mood. Our neural network analyzes the mood or tone of the videos. Some advertisers don’t want to see their ads in videos of a certain tone. We, in turn, can recommend to them the right videos that fit their profile and those that don’t,” – says Vitaly Sotnikov.
What is tone? Vitaly brings up an actual case. Imagine there is a video of a monk self-immolating himself in front of a temple. The network identifies a monk and fire, but it also identifies a “temple”, which lets it conclude that the tone of the video is “religion”. Not all advertisers choose to be affiliated with religion. In a similar manner, many companies choose to avoid politics, terrorism, pedophilia and other unsuitable topics. The NativeOS algorithm can identify these themes on its own, meaning it won’t need 10 thousand employees helping it. Moreover, the context recognition system can be used to let companies know which videos are most likely to be seen by their target audience. This means that customers will see ads they actually like to see, and female geeks watching tech and science videos won’t be subjected to lipstick commercials just because they’re women.
“Native advertising starts with matching, i.e. when advertising fits the content and the customer’s needs and will actually be useful. We were actually surprised once we realized that nobody had tried to automate native ads in videos before us. All the required technology already exists – we’re just the first to put it to use,” – says the NativeOS CEO.
Technology
To perfect the algorithm, developers use freeze-frames taken from clips and movies. The initial teaching process for the neural networks took 15 months. The algorithm also employs computer vision technologies; but it’s not enough to simply identify the objects in the frame. Semantic technologies are used to help the program understand what connects these objects and what these connections mean.
What are semantic technologies? Instead of mathematical methods, they use metadata, i.e. data that describes other data, to describe items and form connections between them. Semantic connections are made using ontologies. These are files or documents that define the relations between various concepts which serve as objects of semantic connections. But it’s important to keep in mind that these connections must remain constant and are set by a person tuning the semantic network (read more about the use of semantic technology in medicine and finance here and here).
“It’s silly to just run a commercial for ties just because the person in the video is wearing one without knowing the context and content of the video,” – explains Mr. Sotnikov.
NativeOS is currently working on improving their neural and semantic networks by improving the number of tags it ascribes to videos of different types. Right now, the company is focused only on image analysis, but among their future plans is to conduct R&D on sound recognition. However, it is rare that sound and image do not match in terms of tone and content, notes Vitaly, so, for the majority of cases, image analysis is sufficient.
There are two features the startup is particularly proud of. The first is the Index of Native Advertising, which determines just how fitting an advertisement is for a particular item. The second is an algorithm that lets native advertising be embedded into videos in a way that it won’t be removed by AdBlock and similar utilities. Both of these technologies are the company’s know-hows and are currently under a strict NDA.
Developers also plan to create a toolset to be used to evaluate the conversion rates of ads embedded using NativeOS. But first, they have to collect enough big data on the program’s use and the number of times users have clicked on the ads. For that purpose, the service needs to be launched and tested on hundreds of ad campaigns.
Working in the USA
It will take another six months or so before the startup launches its product in full. A prototype is currently being tested with the help of a partner – one of the world’s largest communication groups Publicis Groupe. NativeOS is expected to be released on the US market.
“The reason is that people in the USA are more open to new things, quicker to start testing innovative ideas. We tried working with several advertising agencies here in Russia, but it took us half a year just to explain to their managers how our system works. Then there were meetings, planning, contracts… In America, people approach things more openly and simply. For example, we had a meeting with the strategy director of one major digital agency that works with Apple, Google, Facebook and the like. Someone had told him about us and he wanted to meet us and learn more. A couple months later we signed the paperwork and now he’s working with us. This is a good example of how communications happen in the US. There is still bureaucracy, of course, but the people themselves are much more prepared for change, as is the market. The Russian advertising market is falling behind by 5 or 6 years,” – expland Vitaly Sotnikov.
NativeOS’s headquarters are in the New York City. According to the startup’s CEO, it’s a great city for business with lots of opportunities, but it is not the most pleasant place to live.
“Citizens of St. Petersburg often dislike Moscow because it has no cohesive style, it has a different pace. St. Petersburg displays a kind of architectural calm – in the historical part of the city, no building has been built taller than the Winter Palace for a long time. Well, New York is Moscow, but twice as tall and twice as dirty,” – he says.
The time zone differences are also an issue, as the startup’s team and its partners work from all over the world. The language barrier was an obstacle at first, but they adapted quickly.
“Still, we’re genuinely pleased with what we do. I’m also motivated by a long-time wish to create my own TV series; I’ve been polishing a script for four years now, but production costs a lot. And anyway, if we have the chance to change advertising for the better – we should at least try,” – says Sotnikov.
