Debugging and optimizing gstreamer scripts using live streaming as an example

Gstreamer is the most popular * nix video library, which is the basis for most popular applications (the main list can be found here ).

image

However, such a flexible tool requires a delicate approach for stable and efficient operation.
All actions will be considered on the example of online HD broadcasting from HDMI input via RTMP using blackmagic intensity pro, Debian Linux, gstreamer and Nginx.

Debugging ( manual )


Gstreamer has a built-in debugging feature, help can be obtained with the command:
gst-launch --gst-debug-help

The output of debugging information itself is included by adding to the command:
--gst-debug-level=LEVEL
where LEVEL is a number from 0 (without displaying debugging information) to 9 (displaying everything).

Very useful is the option
--gst-debug=GST_CAPS:4
, which will list the possible formats of pipeline elements if two elements cannot find suitable formats for each other.

However, the main problem is the huge amount of information that needs to be interpreted somehow in order to further optimize.
In this situation, there is a great opportunity to generate gstreamer pipeline circuits, which will allow you to visually see the operation order of your pipeline:
  • To generate a .DOT file and further conversion, you need to install the graphviz package .
    apt-get install graphviz
    
  • Next, create an environment variable:
    export GST_DEBUG_DUMP_DOT_DIR=/tmp/dot
    mkdir -p $GST_DEBUG_DUMP_DOT_DIR
    
  • Run Gstreamer with your pipeline, in the / tmp / dot folder .dot files will be created for each state of the pipeline.
  • Convert the received files to a format convenient for you:
    dot -Tpng input.dot > output.png
    dot -Tsvg input.dot > output.svg
    
    Using SVG will allow you to infinitely scale the resulting scheme, since there is enough information on it.
    Example circuit.

According to the data obtained, it will be possible to find the bottlenecks of your conveyor and achieve stable operation.

Optimization


As an example of optimization, the following requirements will be considered:
  • 1920x1080 Full HD Stream Resolution
  • 30 frames per second
  • H264 codec with 768kbit / s bitrate
  • flv container transmitted over RTMP

The goal is to operate the system in real time without frame loss and audio / video synchronization.

Frames / sec ( fps )

It is very important to immediately find out how many frames per second are required and how much your equipment can deliver.
Optimal for a dynamic image is 30fps, but for the transfer of static content (for example, presentation), it makes sense to reduce fps to the minimum comfortable.

Multithreading (using queue )

In Gstreamer, there is a queue element that lives up to its name - this is the data queue (stream).
Using a queue will allow you to parallelize calculations and buffer information automatically for transmission between elements.
This element is one of the key ones along with queue2 (an analogue of queue with buffering to the hard disk) and tee (splitting a stream into several, for example, to write a stream to disk in parallel), and they all deserve a separate article.

Configuring conveyor elements

Of course, the key point is to configure the elements of the pipeline (audio and video decoder, etc.).
  • Avoid copying buffers whenever possible.
    v4l2src always-copy = false
    filesink enable-last-buffer = false
  • Configuring the encoder video to work in real time
    Critical for online broadcasts
    x264enc tune = zerolatency
    dmaienc_h264 encodingpreset = 2

  • Using a variable bitrate for fast operation of the encoder
    Performance gain - up to 3 times
    x264enc pass = 17
    dmaienc_h264 ratecontrol = 2



Practice


An example of an optimized minimum pipeline:
gst-launch \
audiotestsrc ! queue ! audioresample ! voaacenc bitrate=64 ! audio/mpeg,rate=22050,channels=1 ! \
flvmux streamable=1 name=mux \
videotestsrc ! queue ! videorate ! videoscale ! x264enc bitrate=768 tune=zerolatency pass=17 ! \
mux. mux. ! \
rtmpsink location="rtmp://localhost/live/test"

In this case, we are preparing a stream with a video bitrate of 768 and one channel audio bitrate of 64 kbps, respectively, using a queue and converting video with a variable bitrate.
See the conveyor diagram here .

The above example does not take into account some requirements of the task, the final pipeline:
gst-launch \
audiotestsrc ! queue ! audioresample ! audio/x-raw-int,width=16,depth=16,channels=2,rate=22050 ! \
audioconvert ! voaacenc bitrate=64 ! audio/mpeg,rate=22050,channels=1 ! \
flvmux streamable=1 name=mux \
videotestsrc ! queue ! videorate ! videoscale !  ffmpegcolorspace ! \
'video/x-raw-yuv, format=(fourcc)UYVY, framerate=(fraction)30/1' ! \
ffmpegcolorspace ! x264enc bitrate=768 tune=zerolatency pass=17 ! \
mux. mux. ! \
rtmpsink location="rtmp://localhost/live/test"

In this pipeline, it is assumed that videotestsrc gives 1080p.
This example is real and is used on a server with 16 core processor and a Blackmagic intensity pro pci board, the total load is 0.5 LA.
In both examples, the web server used Nginx with the excellent user module rarutyunyan nginx-rtmp-module

Of course, there are many other tools and features for debugging and optimizing Gstreamer,
I will be glad for your examples!

Continuous broadcasts to you, colleagues!

Sources and links:



Also popular now: