June 30, 2017 at 11:35Build IPTV / OTT service: content protection
In this article I want to talk about how video content is protected, and what technologies are used for this. This will mainly be about Internet broadcasting, but you will have to touch on DVB and Multicast, so that it’s more clear what the difference is.
Stalker Middleware, which we installed in a previous article , has integration with our content protection system, as well as with NGINX X-accel and Secure Link.
The article is designed not only for professionals, but also for those who still do not know anything about IPTV / OTT.
There are two approaches to broadcasting video content:
The key difference is the presence of feedback. There are hybrid systems, forget about them as part of this article.
Our company works with broadcasting video over the Internet, so here it’s short, just to make it clearer, what is the difference between scrambling the video at the source and the controlled distribution of the video.
I got into Multicast in Broadcast, because, firstly, it is a form of broadcasting, and secondly, I am not only talking about the transmission of packets on IP networks, but also affect DVB.
With Broadcast broadcasting, there is no feedback between the source and the client, so there is only one way to protect the content - scramble everything on the source. For a client to be able to view protected content, he needs to find out a key that changes periodically. Keys are sent to customers along with the content in such a way that only the recipient can decrypt it. As a rule, they useconditional access modules (CAM modules), and such a system is called a conditional access system .
Almost everyone who uses the services of pay TV has encountered this way of protecting content. For many years, satellite broadcasters have been using this method of protection, forcing them to periodically change access cards and / or CAM modules (or even receivers if the CAM module is built-in).
The main disadvantage of such systems is that keys can be shared ( Card sharing ). Without feedback, you cannot verify the authenticity of a card or module, and it’s technically impossible to find out about the existence and number of intruders.
In IPTV networks, special software is used instead of CAM modules, it communicates with the CAS server and receives keys. This approach virtually eliminates content theft, as communication with the CAS server occurs via a secure Unicast connection, and the receiver does not have a card or module that can be tricked or faked. Managed network equipment allows you to limit the number of channels viewed at the same time (as a rule, operators allow viewing 2-6 channels at the same time), so all content is problematic, as is done with satellite / cable broadcasting.
There is nothing to talk about. Technology has not fundamentally changed for many years. Old systems hack, new ones appear. There are drawbacks, but in general the system works.
With Unicast broadcasting, you can use the Broadcast approach: scramble the stream at the source, and distribute the keys via the Internet. In this case, everything is the same as that of IPTV, but we broadcast not multicast, but via HTTP.
And you can restrict access to content in another way: use tokens unique to each user. The idea is that Middleware gives each user unique links for viewing, and the video server checks these tokens through the Middleware API.
Such a scheme requires fewer servers and software, which gives good savings both at the start and during operation. The content itself is not encrypted, but you cannot access it without permission from Middleware. The portal issues one-time links, the interception of which does not matter - only the device that requested the link can open the link. And HTTPS saves you from banal traffic interception.
“But the content remains open, nothing prevents it from being saved to disk or relayed to other users.” - you say.
Yes, the end user can save segments to his disk and even organize the broadcast to other users. But, one-time links have a lifetime. Set up and forget will not work. You will have to regularly receive new links from Middleware. And in the statistics it will be seen that the user 24/7 is watching the same channel. Suspiciously.
Since the distribution of content over HTTP is a controlled process, that means we can track the number of simultaneous connections. And this means that you can’t watch two channels at the same time (unless, of course, the tariff plan allows it).
To relay all your content, you will have to register the number of accounts equal to the number of TV channels and configure bots that will receive current temporary links emulating the work of the set-top box. Here you have to show imagination, otherwise it will be easy to calculate such users by the logs. Just imagine, from one data center (/ 24 network) 100 users appeared who 24/7 watch the same channels. You will have to scatter such a botnet into different data centers and set up some kind of channel rotation system between accounts so that it is not suspicious. You can discuss in the comments how to steal content and go unnoticed.
“But TV channels do not allow broadcasting them in cable / local / Internet networks without encryption with a certified CAS system.” - you add.
We will not deal with legal nuances within the framework of this article. Different TV channels have different requirements, there are channels that allow open broadcasting, in addition to TV channels, our customers broadcast video from surveillance cameras, which also need to be protected from prying eyes.
The authorization system can be used to transfer video between servers, data centers if you are a content aggregator, and not broadcast directly to customers.
Many customers come to us precisely for the authorization system.
Stalker has several built-in mechanisms for protecting video. They have a number of restrictions: they are not compatible with some protocols (for example, RTSP), cannot fully protect HLS streams and do not take into account simultaneous connections. But they do not require a specialized video server.
If you enable temporary links in the TV channel settings, but don’t check the “NGINX secure link” or “Flussonic support” checkbox, then Stalker will use X-Accel-Redirect to access the content.
This configuration is suitable for protecting HTTP MPEG-TS streams, as only one TCP connection is established between the server and the client. Let's see what the NGINX configuration looks like:
Stalker generates a link of the form: stalker / ch / TOKEN123 , where TOKEN123 is a unique, one-time password. When the client opens this link, NGINX rewrite the file chk_tmp_tv_link.php, which checks the validity of the token and returns a link to the stream using the X-Accel-Redirect header.
Download source package chk_tmp_tv_link.php:
According to the documentation, to use temporary links, you must specify the channel URL in the format: 192.168.1.1 : 8888 / 127.0.0.1: 8899 / udp / 239.1.1.1: 1234, where 192.168.1.1:8888 is the udpxy server with NGINX installed.
After checking the token, Stalker returns a link to stalker / get / 127.0.0.1 : 8899 / udp / 239.1.1.1: 1234 in the $ result variable .
As we know from the NGINX configuration, / get / is an internal location, which means that access here can only be accessed through the X-Accel-Redirect header received from the backend.
Next, using a simple regular expression, NGINX begins to proxy (proxy_pass) to the local (or remote) udpxy.
As you can see, protecting content from unauthorized access is easy. One rewrite, one location with the internal parameter and a small backend script - all this is accessible “out of the box” and works fine. You can read more about running X-Accel in the official NGINX documentation .
NGINX Secure links in Stalker
Now check the “NGINX secure link” box
Limitations:
Stalker protects access to HTTP MPEG-TS streams and links to m3u8 playlists. Chunks and media playlists themselves remain unprotected and with tcpdump / wireshark you can easily find out the addresses of media playlists and connect directly to them.
The easiest and most importantly more reliable way to protect streams is to enable integration with Flussonic. Setting up from the side of the Stalker is not required, you just need to check the “Flussonic” box in the channel settings, and from the side of Flussonic you only need to specify the address of the Stalker.
Flussonic is a video server of our development, widely used in many OTT and IPTV services. Our strengths are access control and session accounting, archive recording. More details on our website .
We have implemented a mechanism for user identification and connection tracking using authorization backends. HLS and HDS protocols use HTTP session tracking mechanisms, and RTMP, RTSP, and MPEG-TS protocols handle persistent TCP sessions. Archive export in MPEG-TS and MP4 format is also monitored.
Stalker - this is the authorization backend. When the client comes for the video, we pass the backend not only the IP address and channel name, but also other important information:
- Token
- HTTP Referer
- The number of open sessions on this stream
- The total number of open sessions on the server
- The requested protocol: (hls, dash, hds, rtmp, rtsp, mpegts or mp4)
- Connection type (new connection or current extension )
- Type of requested content (live stream, archive, screenshots or API call)
- User-Agent
- query string
- Server address and port where the request came from
This information allows the backend to flexibly control access. You can use all the data to build complex rules.
In response, Stalker returns not only yes / no, but also information about the validity period of the permission (or ban), the user id and the number of simultaneous connections for this user.
Flussonic saves the response and does not send repeated requests to the backend for the duration of the permission. Since the article is so lengthy, I won’t show how to configure authorization in Flussonic via Stalker, it’s very simple and we have short, but exhaustive documentation on this issue.
In most cases, to protect content from unauthorized viewing on the Internet, you can do without encryption, this is done easily by free means of the popular Middleware, or you can configure integration with a video server, removing the unnecessary role and load from Stalker.
Tell us in the comments what means of content protection do you use? It is interesting not only about TV broadcasting, but also VOD services, video surveillance.
Judging by the number of views in the first article of the “Building OTT / IPTV Service” series, not everyone has built their solutions and I should not delay the following articles.
Next, we should talk about inserting video on the site, this question is not as simple as it might seem. Read our article, “ What is HTML5 Streaming (and why mp4 Streaming Doesn't Exist)", she talks about modern means of transmitting and viewing streaming video in a browser. And we also launched a free statistics collection service for all our customers.
Stalker Middleware, which we installed in a previous article , has integration with our content protection system, as well as with NGINX X-accel and Secure Link.
The article is designed not only for professionals, but also for those who still do not know anything about IPTV / OTT.
There are two approaches to broadcasting video content:
- Broadcast - when a data stream is uncontrollably spreading over the network. Examples: satellite / cable broadcasting, multicast.
- Unicast - the client comes to the server and requests content. It is used in OTT services.
The key difference is the presence of feedback. There are hybrid systems, forget about them as part of this article.
Broadcast
Our company works with broadcasting video over the Internet, so here it’s short, just to make it clearer, what is the difference between scrambling the video at the source and the controlled distribution of the video.
I got into Multicast in Broadcast, because, firstly, it is a form of broadcasting, and secondly, I am not only talking about the transmission of packets on IP networks, but also affect DVB.
With Broadcast broadcasting, there is no feedback between the source and the client, so there is only one way to protect the content - scramble everything on the source. For a client to be able to view protected content, he needs to find out a key that changes periodically. Keys are sent to customers along with the content in such a way that only the recipient can decrypt it. As a rule, they useconditional access modules (CAM modules), and such a system is called a conditional access system .
Almost everyone who uses the services of pay TV has encountered this way of protecting content. For many years, satellite broadcasters have been using this method of protection, forcing them to periodically change access cards and / or CAM modules (or even receivers if the CAM module is built-in).
The main disadvantage of such systems is that keys can be shared ( Card sharing ). Without feedback, you cannot verify the authenticity of a card or module, and it’s technically impossible to find out about the existence and number of intruders.
In IPTV networks, special software is used instead of CAM modules, it communicates with the CAS server and receives keys. This approach virtually eliminates content theft, as communication with the CAS server occurs via a secure Unicast connection, and the receiver does not have a card or module that can be tricked or faked. Managed network equipment allows you to limit the number of channels viewed at the same time (as a rule, operators allow viewing 2-6 channels at the same time), so all content is problematic, as is done with satellite / cable broadcasting.
There is nothing to talk about. Technology has not fundamentally changed for many years. Old systems hack, new ones appear. There are drawbacks, but in general the system works.
Unicast
With Unicast broadcasting, you can use the Broadcast approach: scramble the stream at the source, and distribute the keys via the Internet. In this case, everything is the same as that of IPTV, but we broadcast not multicast, but via HTTP.
And you can restrict access to content in another way: use tokens unique to each user. The idea is that Middleware gives each user unique links for viewing, and the video server checks these tokens through the Middleware API.
Such a scheme requires fewer servers and software, which gives good savings both at the start and during operation. The content itself is not encrypted, but you cannot access it without permission from Middleware. The portal issues one-time links, the interception of which does not matter - only the device that requested the link can open the link. And HTTPS saves you from banal traffic interception.
“But the content remains open, nothing prevents it from being saved to disk or relayed to other users.” - you say.
Yes, the end user can save segments to his disk and even organize the broadcast to other users. But, one-time links have a lifetime. Set up and forget will not work. You will have to regularly receive new links from Middleware. And in the statistics it will be seen that the user 24/7 is watching the same channel. Suspiciously.
Since the distribution of content over HTTP is a controlled process, that means we can track the number of simultaneous connections. And this means that you can’t watch two channels at the same time (unless, of course, the tariff plan allows it).
To relay all your content, you will have to register the number of accounts equal to the number of TV channels and configure bots that will receive current temporary links emulating the work of the set-top box. Here you have to show imagination, otherwise it will be easy to calculate such users by the logs. Just imagine, from one data center (/ 24 network) 100 users appeared who 24/7 watch the same channels. You will have to scatter such a botnet into different data centers and set up some kind of channel rotation system between accounts so that it is not suspicious. You can discuss in the comments how to steal content and go unnoticed.
“But TV channels do not allow broadcasting them in cable / local / Internet networks without encryption with a certified CAS system.” - you add.
We will not deal with legal nuances within the framework of this article. Different TV channels have different requirements, there are channels that allow open broadcasting, in addition to TV channels, our customers broadcast video from surveillance cameras, which also need to be protected from prying eyes.
The authorization system can be used to transfer video between servers, data centers if you are a content aggregator, and not broadcast directly to customers.
Many customers come to us precisely for the authorization system.
Alright, that's enough theory, let's get into practice.
Stalker has several built-in mechanisms for protecting video. They have a number of restrictions: they are not compatible with some protocols (for example, RTSP), cannot fully protect HLS streams and do not take into account simultaneous connections. But they do not require a specialized video server.
If you enable temporary links in the TV channel settings, but don’t check the “NGINX secure link” or “Flussonic support” checkbox, then Stalker will use X-Accel-Redirect to access the content.
This configuration is suitable for protecting HTTP MPEG-TS streams, as only one TCP connection is established between the server and the client. Let's see what the NGINX configuration looks like:
server{
listen 0.0.0.0:8888;
rewrite ^/ch/(.*) /stalker_portal/server/api/chk_tmp_tv_link.php?key=$1 last;
location /stalker_portal {
internal;
proxy_set_header Host 192.168.1.1; # <-- имя хоста с порталом или IP
proxy_set_header X-Real-IP $remote_addr;
proxy_pass http://192.168.1.1:88/stalker_portal; # <-- IP адрес портала
}
location ~* ^/get/(.*?)/(.*) {
internal;
set $upstream_uri $2;
set $upstream_host $1;
set $upstream_url http://$upstream_host/$upstream_uri;
proxy_set_header Host $upstream_host;
proxy_set_header X-Real-IP $remote_addr;
proxy_pass $upstream_url;
}
}Stalker generates a link of the form: stalker / ch / TOKEN123 , where TOKEN123 is a unique, one-time password. When the client opens this link, NGINX rewrite the file chk_tmp_tv_link.php, which checks the validity of the token and returns a link to the stream using the X-Accel-Redirect header.
Download source package chk_tmp_tv_link.php:
According to the documentation, to use temporary links, you must specify the channel URL in the format: 192.168.1.1 : 8888 / 127.0.0.1: 8899 / udp / 239.1.1.1: 1234, where 192.168.1.1:8888 is the udpxy server with NGINX installed.
After checking the token, Stalker returns a link to stalker / get / 127.0.0.1 : 8899 / udp / 239.1.1.1: 1234 in the $ result variable .
As we know from the NGINX configuration, / get / is an internal location, which means that access here can only be accessed through the X-Accel-Redirect header received from the backend.
Next, using a simple regular expression, NGINX begins to proxy (proxy_pass) to the local (or remote) udpxy.
As you can see, protecting content from unauthorized access is easy. One rewrite, one location with the internal parameter and a small backend script - all this is accessible “out of the box” and works fine. You can read more about running X-Accel in the official NGINX documentation .
NGINX Secure links in Stalker
Now check the “NGINX secure link” box
Limitations:
Stalker protects access to HTTP MPEG-TS streams and links to m3u8 playlists. Chunks and media playlists themselves remain unprotected and with tcpdump / wireshark you can easily find out the addresses of media playlists and connect directly to them.
The easiest and most importantly more reliable way to protect streams is to enable integration with Flussonic. Setting up from the side of the Stalker is not required, you just need to check the “Flussonic” box in the channel settings, and from the side of Flussonic you only need to specify the address of the Stalker.
Flussonic is a video server of our development, widely used in many OTT and IPTV services. Our strengths are access control and session accounting, archive recording. More details on our website .
We have implemented a mechanism for user identification and connection tracking using authorization backends. HLS and HDS protocols use HTTP session tracking mechanisms, and RTMP, RTSP, and MPEG-TS protocols handle persistent TCP sessions. Archive export in MPEG-TS and MP4 format is also monitored.
Stalker - this is the authorization backend. When the client comes for the video, we pass the backend not only the IP address and channel name, but also other important information:
- Token
- HTTP Referer
- The number of open sessions on this stream
- The total number of open sessions on the server
- The requested protocol: (hls, dash, hds, rtmp, rtsp, mpegts or mp4)
- Connection type (new connection or current extension )
- Type of requested content (live stream, archive, screenshots or API call)
- User-Agent
- query string
- Server address and port where the request came from
This information allows the backend to flexibly control access. You can use all the data to build complex rules.
In response, Stalker returns not only yes / no, but also information about the validity period of the permission (or ban), the user id and the number of simultaneous connections for this user.
Flussonic saves the response and does not send repeated requests to the backend for the duration of the permission. Since the article is so lengthy, I won’t show how to configure authorization in Flussonic via Stalker, it’s very simple and we have short, but exhaustive documentation on this issue.
To summarize.
In most cases, to protect content from unauthorized viewing on the Internet, you can do without encryption, this is done easily by free means of the popular Middleware, or you can configure integration with a video server, removing the unnecessary role and load from Stalker.
Tell us in the comments what means of content protection do you use? It is interesting not only about TV broadcasting, but also VOD services, video surveillance.
Judging by the number of views in the first article of the “Building OTT / IPTV Service” series, not everyone has built their solutions and I should not delay the following articles.
Next, we should talk about inserting video on the site, this question is not as simple as it might seem. Read our article, “ What is HTML5 Streaming (and why mp4 Streaming Doesn't Exist)", she talks about modern means of transmitting and viewing streaming video in a browser. And we also launched a free statistics collection service for all our customers.