Smart TV, which itself determines the most interesting channel, or an unusual solution to sudoku for video content
How do you think monetization of TV channels, neural networks and sudoku are related? At first glance, it is like programming with art: perhaps, but not many succeed. However, after looking closely, our team found an unusual answer to this question. The product, which was developed by engineers, allowed using different types of neural networks to determine the beginning and end of ad units so that they appeared at the right time. Now, he has already transformed into a universal tool for solving a wide range of tasks. And we are happy to tell you what's what.
It all started with the fact that monetization of content through advertising was the main source of profit for American streaming companies and TV channels. The owners of TV channels allowed providers to replace a certain part of the advertisement and display ads of their clients there. However, if the ad units overlapped each other, the advertisers left negative feedback. As a result, advertising on the channel was turned off, and income was lost.
To solve this problem, different approaches were used. One of the largest suppliers of video content in the United States, which began working with EPAM, had its own approach. However, its accuracy did not exceed sixty percent. This means that in forty percent of cases, commercials overlapped each other, which could lead to a complete disconnection of advertising on these TV channels.
According to project manager Vlad Bayovskiy , “as soon as the project started, engineer Lesha Druzhinin ran into this problem, and he had an idea. It consisted in creating a product that would determine the beginning and end of ad units. Frankly, at first I was skeptical, because the solution was technically difficult, that is, it involved the use of neural networks and machine learning. And at that time we did not have the appropriate expertise in these areas.
In general, all the guys who managed to take part in this project, tried different types of neural networks and different algorithms, connected and synchronized different components. Despite the fact that the idea was born in California, we managed to use 70 engineers from completely different locations of EPAM.
As a result, we have an excellent product, which now not only determines the change of advertising, but is a universal solution for any tasks related to determining the change of video content, including a potential solution for video search. In other words, its scope is not limited to advertising - it can be, for example, a TV show, which changes headings, and data from video cameras or digitized old movies that do not have metadata about the content of the video sequence. ”
Lesha Druzhinin , Senior Software Engineer:
“If we didn’t do the project, but put on a play, we would have made a very good play! Her genres are constantly changing, there is a cardinal and frequent change of actions, there is intrigue. At first we tried to solve the problem in a simple way, without neural networks, but nothing happened. We had to completely deal with the device of most types of neural networks and train them from zero to perform certain tasks. To make it clear - there are many neural network architectures, but we were interested in those that process visual information.
Over time, we noticed that our project became the champion in the company using neural networks: we had more than five types of them. Now we stopped at two. They analyze every frame and every piece of sound. When a cardinal color change of frame or tone of sound occurs, the networks react to this by predicting the end of the advertisement and then the transition between the ad units. So far, the network that defines the transition between the rollers, more precisely, the one that predicts the end of the video.
Each of them gets its copy of the cut frames and sounds, but together they provide 90 percent accuracy of the result. We launched this solution on real channels, and it works. We managed to create a bot that can watch TV. Figuratively speaking, if after this project the clients come to us with the request “And make us an improved Smart TV”, then we are ready for such tasks. Our solution - this is a small Smart TV, which is already able to independently distinguish content, only for a specific niche. ”
By the way, about the experience of world IT-companies. While working on this project, the EPAM team came to the conclusion that global corporations have been practicing for a long time - requirements for open source. Since changes to the project occurred very often and new engineers constantly connected, the code simply needs to be flexible. Therefore, constant checks of its readability or, as they say, meaty were carried out.
Initially, the team’s task was to “hit, surprise and win the customer’s trust”. However, in the process of working on the solution, the starting concept has improved to such an extent that a universal product has turned out, which after a certain refinement can be offered to many companies whose business is related to video content.
Worked on the article: Daria Prokopovich, Olga Nechaeva
It all started with the fact that monetization of content through advertising was the main source of profit for American streaming companies and TV channels. The owners of TV channels allowed providers to replace a certain part of the advertisement and display ads of their clients there. However, if the ad units overlapped each other, the advertisers left negative feedback. As a result, advertising on the channel was turned off, and income was lost.
To solve this problem, different approaches were used. One of the largest suppliers of video content in the United States, which began working with EPAM, had its own approach. However, its accuracy did not exceed sixty percent. This means that in forty percent of cases, commercials overlapped each other, which could lead to a complete disconnection of advertising on these TV channels.
According to project manager Vlad Bayovskiy , “as soon as the project started, engineer Lesha Druzhinin ran into this problem, and he had an idea. It consisted in creating a product that would determine the beginning and end of ad units. Frankly, at first I was skeptical, because the solution was technically difficult, that is, it involved the use of neural networks and machine learning. And at that time we did not have the appropriate expertise in these areas.
In general, all the guys who managed to take part in this project, tried different types of neural networks and different algorithms, connected and synchronized different components. Despite the fact that the idea was born in California, we managed to use 70 engineers from completely different locations of EPAM.
As a result, we have an excellent product, which now not only determines the change of advertising, but is a universal solution for any tasks related to determining the change of video content, including a potential solution for video search. In other words, its scope is not limited to advertising - it can be, for example, a TV show, which changes headings, and data from video cameras or digitized old movies that do not have metadata about the content of the video sequence. ”
Lesha Druzhinin , Senior Software Engineer:
“If we didn’t do the project, but put on a play, we would have made a very good play! Her genres are constantly changing, there is a cardinal and frequent change of actions, there is intrigue. At first we tried to solve the problem in a simple way, without neural networks, but nothing happened. We had to completely deal with the device of most types of neural networks and train them from zero to perform certain tasks. To make it clear - there are many neural network architectures, but we were interested in those that process visual information.
Over time, we noticed that our project became the champion in the company using neural networks: we had more than five types of them. Now we stopped at two. They analyze every frame and every piece of sound. When a cardinal color change of frame or tone of sound occurs, the networks react to this by predicting the end of the advertisement and then the transition between the ad units. So far, the network that defines the transition between the rollers, more precisely, the one that predicts the end of the video.
Each of them gets its copy of the cut frames and sounds, but together they provide 90 percent accuracy of the result. We launched this solution on real channels, and it works. We managed to create a bot that can watch TV. Figuratively speaking, if after this project the clients come to us with the request “And make us an improved Smart TV”, then we are ready for such tasks. Our solution - this is a small Smart TV, which is already able to independently distinguish content, only for a specific niche. ”
By the way, about the experience of world IT-companies. While working on this project, the EPAM team came to the conclusion that global corporations have been practicing for a long time - requirements for open source. Since changes to the project occurred very often and new engineers constantly connected, the code simply needs to be flexible. Therefore, constant checks of its readability or, as they say, meaty were carried out.
Initially, the team’s task was to “hit, surprise and win the customer’s trust”. However, in the process of working on the solution, the starting concept has improved to such an extent that a universal product has turned out, which after a certain refinement can be offered to many companies whose business is related to video content.
Worked on the article: Daria Prokopovich, Olga Nechaeva