October 6, 2015 at 11:05Flussonic Video Streamer Overview
For several years of programming, we have added a large number of different features to Flussonic, so I want to talk a little about how Flussonic works and what it can do.
To begin with, let me remind you: Flussonic is a state of art video streaming server program that can take video from various sources, burn it to disk as needed and distribute it to clients using various protocols. It can work on Intel, ARM, Tilera, etc. A distinctive feature is the special ease of configuration, which does not require a single megabyte of XML to configure and run.
In the internal structure and functionality, two large and important parts can be distinguished: working with video streams and working with video files (VOD).
VOD - video on demand. This marketing abbreviation came from far depths, when the very idea that a user can take and request a video from the server when he wants, and not according to a schedule, seemed somehow unbelievably cool. At some stage in the development of computers and networks, it was really quite cool (the networks are thin, the video is thick, not enough for all LANs), although today it already causes a slight smile: just think, I’ve never seen a video from the Internet.
Talk less with files, so go ahead. Flussonic can read files in which, in mp4 (mov) or, in the worst case, flv container, video and audio are recorded in the h264, hevc (h265), aac, ac3, mp3 codecs and send this video to clients using rtmp, rtsp, HLS, HDS, DASH
An important difference from ffmpeg is that Flussonic does the so-called segmentation on the fly. Some VOD streaming systems require preliminary preparation, for example, video segmentation (a bunch of mpegts segments for HLS are made from a mp4 file) or video preprocessing to save a nearby index for rewinding.
Those. Flussonic works very conveniently for the administrator: put the files and they are available. Moreover, unlike ordinary browser pseudo-streaming, there is no need to process files and move the header of the mp4 file (moov atom) to the beginning. We spent a huge amount of time and effort to make reading headers in mp4 so fast that Flussonic can send many, many gigabits per second from one server in this segmentation mode on the fly.
In especially loaded systems where it is necessary to assemble a combined HDD + SSD configuration or the traffic exceeds 8-10 gigabits, Flussonic can use the segment cache, but this is a topic for a separate story, as is our separate queue mechanism for different drives. You can also configure several different paths for finding a file when, for whatever reason, you do not want to collect several disks in RAID.
When one server is not enough to store all (of course legally obtained) movies, Flussonic can take video over HTTP in pieces. Those. if the user came to the edge server with the desire to watch a 3-hour movie, then the viewing will begin almost immediately, and not when the entire film is copied to the server. You can take it not only from a simple HTTP server, but also from Amazon S3 (and compatible) and various Swift (OpenStack Storage) storage options.
We have a VOD manager with download, which allows you to work with all file placement options (including multi-paths). You can upload many files at once, create directories (including on Swift / S3) and upload files. Transcoding downloaded files is in question, this feature is temporarily turned off.
A file in which there are several audio tracks and subtitles, Flussonic will be able to give the main players so that they allow you to select the language and subtitles. If the file also has several video tracks, then the user will get the option of adaptive multi-bitrate.
If your content provider twists you under the contract and requires content encryption, then Flussonic can prepare video for Verimatrix or Conax client. But this is not the most fun topic, is it?
Working with video streams is a much more voluminous, problematic and rich part of our work than working with files.
Video stream is a large and bold entity inside Flussonic, which collects up to a dozen different Erlang processes under it, receives video from the input, optionally transcodes (more on this below), if necessary, writes to disk and distributes it to different outputs.
First you probably need to talk about the sources of video streams. Flussonic can act as the initiator of video transmission, and can act as a passive waiting video. In the first case, we are talking about the sources of the video stream, and in the second we are talking about publishing the video to the server. When a stream has sources, Flussonic can itself control when and to which source to turn for video and the algorithm of this control is very difficult. With the publication, you can simply allow or not allow the video to be accepted.
If Flussonic itself initiates the transfer of video, then the video source must be explicitly specified in the configuration file. A stream may have several sources and the behavior will vary depending on the settings.
If the sources are indicated as variants of the same video, but with a different bitrate, then one multi-bitrate stream will be formed in which the client can select the desired quality. For example, you can add several profiles from the IP camera as different qualities for one stream and get a multi-bitrate.
The protocol support is wide: RTSP, MPEG-TS, RTMP, HLS (and not only from fluxonics, but from all servers, not everyone can do that), Shoutcast, managed server playlists and even SDI capture cards.
In another case, Flussonic will switch from the first source to the second and further in the event of a failure of the first and return back after restoration. For example, you can specify multicast from the head station as the first source, and add an http stream from a neighbor as an alternative source.
If there are no sources left, you can specify the stub file as the source for the stream.
You can specify the stream as constantly running (and unlike some other servers, it will be a really 7/24 constantly working stream, which you do not need to monitor), or you can configure it as a stream that starts at the user's request. For example, this happens when many cameras are configured, not all of which are constantly interesting.
Flussonic does a very delicate and fragile job of “stitching” multiple sources when switching. When the source “blinks” or switches to another, the time stamps in the stream go as if nothing had happened.
In normal mode, Flussonic does not change the video itself, only changes the packaging and gives exactly what it received. Sometimes this is not good and you need to change the codec itself (or its parameters such as bit rate), then you need to start transcoding. This is an extremely resource-intensive procedure that reduces the number of threads on a single server from, conditionally, thousands to ten.
Flussonic will be able to transcode on the processor and will soon learn to use Intel's Quicksync and Nvidia nvenc. Several Flussonic in a group can carry out transcoding redundancy, catching streams when one of the servers crashes.
Most often, transcoding is used in IPTV to prepare satellite video for broadcast on the Internet to set-top boxes and mobile devices, but sometimes it is also used for IP cameras.
This is truly our premium feature. Suffice it to say that our archive is used for storing annual video recordings and for broadcasting to 5,000 simultaneous users and it copes with it, while we can use both SSD / HDD and industrial (consider slow and funny) NASs for the archive, available over NFS. And there is something to tinker with!
Our implementation of the archive does not require any action from the user, such as tracking the recording depth or free space on the disk, you do not need to configure any databases, everything is done by us, carefully packed into the tested code and just works. Almost all supported codecs are written to the archive: h264, hevc, aac, mp3, pcma, pcmu. The video is available in the form of an endless tape that can be viewed from any moment.
For stories with IP cameras, we can store information about the operation of a motion detector. The archive is often used for broadcasting with a time shift (relevant for broadcasting Eurasian television channels in North America).
About the archive, we will write some more large detailed articles, so for now we say that we have about 30 different ways to get to the archive, including cluster configurations, when the archive is written in one place and is accessible from all servers.
As in the case of files, Flussonic segments streams on the fly for the HLS, HDS, DASH protocols, and at the same time makes them accessible via the RTMP, RTSP, HTTP MPEG-TS protocols and at the same time is able to send UDP MPEG-TS multicast to the network.
Unlike some other servers, Flussonic stores all segments in memory. The alternative here is burning to disk. If you do not configure tmpfs for temporary storage of segments, then the disk will have to be changed often and sadly. Flussonic keeps everything in memory and makes it possible to get to one stream from all server cores, allowing you to distribute one TV channel at 20 gigabits per second (this is taking into account the sessions, but more on that below).
A huge amount of effort has been invested in the MPEG-TS UDP distribution mechanism, and as a result, this subsystem is able to send video to the network without any jumps, which comes in 10-second blocks via HLS.
To control flows, configure them, etc. we have a web interface with the ability to view stream logs, graphs, edit settings, etc.
We will tell you in more detail about the features of Flussonic, which make it a better choice compared to alternatives, but these are the topics of individual articles.
In addition to distributing videos, Flussonic has a number of other popular features.
We are regularly asked about the possibility of programming inside the server itself and it always turns out that this is not necessary and our HTTP API is enough. Flussonic provides the ability to manage and receive information via HTTP, as well as call external handlers in certain cases: either via HTTP, or using the built-in lua interpreter.
Also, most of the functionality is available through the SQL API . Yes, our Flussonic can pretend to be a Mysql server (oh, how much we did for different ORMs to believe that it was Mysql) and give internal data in response to SQL queries. Through SQL, you can even control threads in the entire cluster . This can be much more convenient than making HTTP requests, for example, from Rails.
We have recorded the sessions in the best possible way, few can compare.
Customers ask us: how many users do I watch videos? When all users are connected via http mpegts or rtmp, there are no difficulties: just see how many open sockets are. Everything is complicated with the transition to HLS, and taking into account all the variability of those cute features that various customers bring with them. It is necessary to glue consecutive HTTP requests in the session and to do this is not easy.
We'll tell you more about authorization in more detail, but now I want to boast of such a cool feature as limiting the number of simultaneous views from one user_id, and this feature works when the user methodically flips through all available 400 channels back and forth in search of interesting things. He works for him, and the cunning ones who decided to steal a package for restriming will be left with nothing.
While we were creating a system for storing graphs in Flussonic, we made our own ultra-compact storage for time series and it is used in every instance of our software.
Flussonic collects and writes a variety of metrics: general information on the network, on loading the CPU, disks, the number of users and detailed metrics for each stream and file. All this information is available in the web interface either in the form of ready-made graphs, or in the form of an interface where you can build queries to the database of metrics using special languages.
Several Flussonic-s are able to cooperate to solve different problems. For example, you can configure several servers to capture a large group of threads in such a mode when, when one of the servers crashes, the rest disassemble its streams until it returns to operation.
You can also conveniently point the edge to the server with a bunch of sources so that he can decide which one of them he can take the stream for which the user came. When there are a couple of thousand threads in the system, it becomes very convenient.
Watcher is our separate product that runs on top of Flussonic. The fact is that in Flussonic itself there are no users. There is only one login, and that administrator’s living person cannot be allowed into the admin panel — only the administrator.
Watcher is a system for managing users and their access rights to IP cameras. In it, you can give rights to users to a group of cameras, add, delete people. There is a mechanism for quickly finding cameras (when you need to add 200 pieces, this is a very useful thing), you can twist a PTZ camera, etc.
We will also talk about him separately.
You can always take the key to the test on our website .
To begin with, let me remind you: Flussonic is a state of art video streaming server program that can take video from various sources, burn it to disk as needed and distribute it to clients using various protocols. It can work on Intel, ARM, Tilera, etc. A distinctive feature is the special ease of configuration, which does not require a single megabyte of XML to configure and run.
In the internal structure and functionality, two large and important parts can be distinguished: working with video streams and working with video files (VOD).
Vod
VOD - video on demand. This marketing abbreviation came from far depths, when the very idea that a user can take and request a video from the server when he wants, and not according to a schedule, seemed somehow unbelievably cool. At some stage in the development of computers and networks, it was really quite cool (the networks are thin, the video is thick, not enough for all LANs), although today it already causes a slight smile: just think, I’ve never seen a video from the Internet.
Talk less with files, so go ahead. Flussonic can read files in which, in mp4 (mov) or, in the worst case, flv container, video and audio are recorded in the h264, hevc (h265), aac, ac3, mp3 codecs and send this video to clients using rtmp, rtsp, HLS, HDS, DASH
An important difference from ffmpeg is that Flussonic does the so-called segmentation on the fly. Some VOD streaming systems require preliminary preparation, for example, video segmentation (a bunch of mpegts segments for HLS are made from a mp4 file) or video preprocessing to save a nearby index for rewinding.
Those. Flussonic works very conveniently for the administrator: put the files and they are available. Moreover, unlike ordinary browser pseudo-streaming, there is no need to process files and move the header of the mp4 file (moov atom) to the beginning. We spent a huge amount of time and effort to make reading headers in mp4 so fast that Flussonic can send many, many gigabits per second from one server in this segmentation mode on the fly.
In especially loaded systems where it is necessary to assemble a combined HDD + SSD configuration or the traffic exceeds 8-10 gigabits, Flussonic can use the segment cache, but this is a topic for a separate story, as is our separate queue mechanism for different drives. You can also configure several different paths for finding a file when, for whatever reason, you do not want to collect several disks in RAID.
When one server is not enough to store all (of course legally obtained) movies, Flussonic can take video over HTTP in pieces. Those. if the user came to the edge server with the desire to watch a 3-hour movie, then the viewing will begin almost immediately, and not when the entire film is copied to the server. You can take it not only from a simple HTTP server, but also from Amazon S3 (and compatible) and various Swift (OpenStack Storage) storage options.
We have a VOD manager with download, which allows you to work with all file placement options (including multi-paths). You can upload many files at once, create directories (including on Swift / S3) and upload files. Transcoding downloaded files is in question, this feature is temporarily turned off.
A file in which there are several audio tracks and subtitles, Flussonic will be able to give the main players so that they allow you to select the language and subtitles. If the file also has several video tracks, then the user will get the option of adaptive multi-bitrate.
If your content provider twists you under the contract and requires content encryption, then Flussonic can prepare video for Verimatrix or Conax client. But this is not the most fun topic, is it?
Live streaming
Working with video streams is a much more voluminous, problematic and rich part of our work than working with files.
Video stream is a large and bold entity inside Flussonic, which collects up to a dozen different Erlang processes under it, receives video from the input, optionally transcodes (more on this below), if necessary, writes to disk and distributes it to different outputs.
First you probably need to talk about the sources of video streams. Flussonic can act as the initiator of video transmission, and can act as a passive waiting video. In the first case, we are talking about the sources of the video stream, and in the second we are talking about publishing the video to the server. When a stream has sources, Flussonic can itself control when and to which source to turn for video and the algorithm of this control is very difficult. With the publication, you can simply allow or not allow the video to be accepted.
Source management
If Flussonic itself initiates the transfer of video, then the video source must be explicitly specified in the configuration file. A stream may have several sources and the behavior will vary depending on the settings.
If the sources are indicated as variants of the same video, but with a different bitrate, then one multi-bitrate stream will be formed in which the client can select the desired quality. For example, you can add several profiles from the IP camera as different qualities for one stream and get a multi-bitrate.
The protocol support is wide: RTSP, MPEG-TS, RTMP, HLS (and not only from fluxonics, but from all servers, not everyone can do that), Shoutcast, managed server playlists and even SDI capture cards.
In another case, Flussonic will switch from the first source to the second and further in the event of a failure of the first and return back after restoration. For example, you can specify multicast from the head station as the first source, and add an http stream from a neighbor as an alternative source.
If there are no sources left, you can specify the stub file as the source for the stream.
You can specify the stream as constantly running (and unlike some other servers, it will be a really 7/24 constantly working stream, which you do not need to monitor), or you can configure it as a stream that starts at the user's request. For example, this happens when many cameras are configured, not all of which are constantly interesting.
Flussonic does a very delicate and fragile job of “stitching” multiple sources when switching. When the source “blinks” or switches to another, the time stamps in the stream go as if nothing had happened.
Transcoding
In normal mode, Flussonic does not change the video itself, only changes the packaging and gives exactly what it received. Sometimes this is not good and you need to change the codec itself (or its parameters such as bit rate), then you need to start transcoding. This is an extremely resource-intensive procedure that reduces the number of threads on a single server from, conditionally, thousands to ten.
Flussonic will be able to transcode on the processor and will soon learn to use Intel's Quicksync and Nvidia nvenc. Several Flussonic in a group can carry out transcoding redundancy, catching streams when one of the servers crashes.
Most often, transcoding is used in IPTV to prepare satellite video for broadcast on the Internet to set-top boxes and mobile devices, but sometimes it is also used for IP cameras.
Archive recording
This is truly our premium feature. Suffice it to say that our archive is used for storing annual video recordings and for broadcasting to 5,000 simultaneous users and it copes with it, while we can use both SSD / HDD and industrial (consider slow and funny) NASs for the archive, available over NFS. And there is something to tinker with!
Our implementation of the archive does not require any action from the user, such as tracking the recording depth or free space on the disk, you do not need to configure any databases, everything is done by us, carefully packed into the tested code and just works. Almost all supported codecs are written to the archive: h264, hevc, aac, mp3, pcma, pcmu. The video is available in the form of an endless tape that can be viewed from any moment.
For stories with IP cameras, we can store information about the operation of a motion detector. The archive is often used for broadcasting with a time shift (relevant for broadcasting Eurasian television channels in North America).
About the archive, we will write some more large detailed articles, so for now we say that we have about 30 different ways to get to the archive, including cluster configurations, when the archive is written in one place and is accessible from all servers.
Video distribution
As in the case of files, Flussonic segments streams on the fly for the HLS, HDS, DASH protocols, and at the same time makes them accessible via the RTMP, RTSP, HTTP MPEG-TS protocols and at the same time is able to send UDP MPEG-TS multicast to the network.
Unlike some other servers, Flussonic stores all segments in memory. The alternative here is burning to disk. If you do not configure tmpfs for temporary storage of segments, then the disk will have to be changed often and sadly. Flussonic keeps everything in memory and makes it possible to get to one stream from all server cores, allowing you to distribute one TV channel at 20 gigabits per second (this is taking into account the sessions, but more on that below).
A huge amount of effort has been invested in the MPEG-TS UDP distribution mechanism, and as a result, this subsystem is able to send video to the network without any jumps, which comes in 10-second blocks via HLS.
To control flows, configure them, etc. we have a web interface with the ability to view stream logs, graphs, edit settings, etc.
We will tell you in more detail about the features of Flussonic, which make it a better choice compared to alternatives, but these are the topics of individual articles.
Related features
In addition to distributing videos, Flussonic has a number of other popular features.
API & scripting
We are regularly asked about the possibility of programming inside the server itself and it always turns out that this is not necessary and our HTTP API is enough. Flussonic provides the ability to manage and receive information via HTTP, as well as call external handlers in certain cases: either via HTTP, or using the built-in lua interpreter.
Also, most of the functionality is available through the SQL API . Yes, our Flussonic can pretend to be a Mysql server (oh, how much we did for different ORMs to believe that it was Mysql) and give internal data in response to SQL queries. Through SQL, you can even control threads in the entire cluster . This can be much more convenient than making HTTP requests, for example, from Rails.
Session Authorization and Accounting
We have recorded the sessions in the best possible way, few can compare.
Customers ask us: how many users do I watch videos? When all users are connected via http mpegts or rtmp, there are no difficulties: just see how many open sockets are. Everything is complicated with the transition to HLS, and taking into account all the variability of those cute features that various customers bring with them. It is necessary to glue consecutive HTTP requests in the session and to do this is not easy.
We'll tell you more about authorization in more detail, but now I want to boast of such a cool feature as limiting the number of simultaneous views from one user_id, and this feature works when the user methodically flips through all available 400 channels back and forth in search of interesting things. He works for him, and the cunning ones who decided to steal a package for restriming will be left with nothing.
Collection of statistics and graphs
While we were creating a system for storing graphs in Flussonic, we made our own ultra-compact storage for time series and it is used in every instance of our software.
Flussonic collects and writes a variety of metrics: general information on the network, on loading the CPU, disks, the number of users and detailed metrics for each stream and file. All this information is available in the web interface either in the form of ready-made graphs, or in the form of an interface where you can build queries to the database of metrics using special languages.
Clustering
Several Flussonic-s are able to cooperate to solve different problems. For example, you can configure several servers to capture a large group of threads in such a mode when, when one of the servers crashes, the rest disassemble its streams until it returns to operation.
You can also conveniently point the edge to the server with a bunch of sources so that he can decide which one of them he can take the stream for which the user came. When there are a couple of thousand threads in the system, it becomes very convenient.
Flussonic watcher
Watcher is our separate product that runs on top of Flussonic. The fact is that in Flussonic itself there are no users. There is only one login, and that administrator’s living person cannot be allowed into the admin panel — only the administrator.
Watcher is a system for managing users and their access rights to IP cameras. In it, you can give rights to users to a group of cameras, add, delete people. There is a mechanism for quickly finding cameras (when you need to add 200 pieces, this is a very useful thing), you can twist a PTZ camera, etc.
We will also talk about him separately.
You can always take the key to the test on our website .