Distribution of video. Ambush: nginx or php?

I want to share my little experience in the implementation of the distribution of video content.

so

There is a service that distributes video content for viewing (i.e., downloading is not provided for). At the same time, all content is divided into 2 categories:

1 - given in its entirety (full file) / pause, rewind;
2 - given one “virtual file” of the form [end of the first file] + [some number of files completely] + [beginning of the last file]. The format is mpegts, each set is encoded the same way, so you can just glue the parts together.

These two categories differ logically (completely different things), have clearly different URIs and are physically located in different places.

As originally suggested

A bunch of nginx + apache.

The first category is the banal distribution of content in nginx with a little tuning.
The second part - through the apache-php script using a loop

while(!feof($fp)){
...
echo fread($fp, $buf)
...
}

where $ fp is a pointer to the file with fseek () executed where needed.

What did not like

As it turned out, nginx is not very suitable for distributing statics with a large number of range-bytes requests (namely, such requests are mainly obtained during online viewing). There is no way to use AIO to serve such requests. As a result, a long queue to disk is formed, and for customers watching a video is often accompanied by “brakes”. Setting a large number of buffers is useless - just wasting memory.

I tried the latest version of nginx (1.12.2 at the moment), and --with-file-aio, and --with-threads, and all sorts of different things. The effect is not received.

Well, the bunch of "echo fread ()" in php is also very dubious. The output of fread goes to the intermediate php buffer and, accordingly, the script consumes memory not less than this buffer. If you read the file in small pieces, then the load on the CPU increases and the upload speed drops. If you read in large chunks, then each request consumes too much memory.

What ultimately happened

Well, first of all I refused apache. Instead php5-fpm. This gave a significant increase in speed (response speed) + reduced memory consumption.

First category

content, for the sake of experiment, I decided to try to give it away with my script.

In nginx:

 location ~* /media/.*\..* {
        fastcgi_pass unix:/var/run/php5-fpm.sock;
        include /etc/nginx/sites-available/fastcgi_params;
        fastcgi_param   SCRIPT_FILENAME     /var/www/m_download.php;
        root /var/www;
        send_timeout 1h;
    }

I will not give m_download.php completely. The main functionality:

fpassthru($fd);

where $ fd is a file pointer. First, of course, you need to parse the HTTP_RANGE header, open the file and set the offset in it.

fpassthru () gives the file "from the current to the end." In this situation, it is quite suitable. All players play correctly.

To my great surprise, this particular method of return gave the result I needed. There is no queue for the disk (more precisely, the system is used, but with my SAS-3 12Gb / sec and await <10ms it’s generally good). Accordingly, there is no waiting for request processing. File upload speed (if downloaded) - about 250 Mbit / s. The "brakes" of the customers are completely gone.

At the same time, memory usage is greatly reduced, therefore more remains for the file cache. The script itself consumes about 0.5 MB of private memory during execution. The executable code still exists in memory in a single instance, so its size does not matter.

Second category

(this is where you need to mold several pieces of different files) has also changed.

Refused a bunch of "echo fread ()".

Unfortunately, in php there is no function to directly output an arbitrary piece of a file. In fpassthru () there is no parameter “how much to output”, it always displays “to the end”.

I tried calling system dd with passthru ().
Those.:

passthru('/bin/dd status=none if='.$fffilename.' iflag="skip_bytes,count_bytes" skip='.$ffseek.' count='.$buf_size);

And ... oh, a miracle! The memory consumption of the script is slightly more than 0.5 MB, I can set any buffer size (it does not affect the memory). The rate of return (with a buffer of 4 MB) ... whistles (the same 250 Mbit / s).

Here is a story. As a result, I had to refuse to distribute content simply nginx. It is used, but only to redirect requests to php5-fpm.

In short, my IMHO: nginx gives good statics, but poorly reads from disk.

I would never have thought that distributing files through a php script might be more efficient.
Well, I’ll add that I was looking for some httpd with AIO for range-bytes of requests “out of the box”. It seems that lighttpd of the 2nd version can, but the version is still unstable ... I did not find anything else suitable.