RTCP REMB: tweak the video call settings in the browser

Original author: Timote Le Bornier
  • Transfer

Resourceful guys from Tribe not only found their niche at the junction of conference and gamedev (which is already interesting in itself), but they also know what’s going on under the hood of WebRTC. Product Manager Timote Le Bornier periodically describes the technical side of his brainchild; we decided to translate his article about the non-standard, but very curious WebRTC functionality. We are sure you will be interested. In the meantime, as you read, we will continue to prepare an insider about our updated video conferencing ... But that's another story.

Fine-tune and control flow quality using REMB

So, now you most likely know that we were working on creating a custom MCU for Tribe - if you don’t know, then that’s why we decided to do it and how we do it .

But closer to the point: let's see what happens under the hood.

  1. WebRTC-connection starts with an exchange mechanism called ”SDP offer / answer” - exchange of information about supported codecs and related parameters between participants.
  2. After that, participants try to communicate with each other directly: they start listening to UDP ports and use ICE, STUN and (later) TURN to establish a UDP connection.
  3. When a UDP connection is established, participants use it to exchange encryption parameters using the DTLS-SRTP protocol. After that, participants use:
    • SRTP (Secure RTP over UDP) protocol to send and receive media
    • SRTCP (Secure RTCP over UDP) for flow control and out-of-band communication.

WebRTC protocol stack (modified schema, original here: webrtc-security.github.io )

Understanding RTCP

It is mentioned above that RTP and RTCP packets are safely stored in SRTP and SRTCP. What are they used for?

RTP provides end-to-end transport for real-time data: audio and video. In a way, RTCP is the “RTP brother”: it is used with RTP and improves the data transfer protocol by monitoring delivery, as well as management and identification.

In a WebRTC session, the MCU acts as an intermediary between participants: they are connected to the MCU, but not directly to each other.

WebRTC - The basic topology of the MCU

For example, take a session with two participants. The MCU receives the media stream from the FOO participant, decodes, encodes it and sends it to the BAR participant. At the same time, the same thing happens with the flow from BAR to FOO.

MTP and RTCP protocols

Symmetry is obvious, so let's focus on one direction and see what is going on there:

Focus on one side

When the BAR browser receives a media stream from the MCU, it also receives statistics from RTCP Sender Reports (SR). The BAR Browser will use RTP's serial numbers, RTP timestamps, native time, and Sender Reports to calculate what we call RTCP Receiver Reports (RR).

The BAR browser uses all this data to calculate the maximum expected bitrate, and then sends this value to the MCU using RTCP REMB (Remote Estimate Maximum Bitrate).

Using REMB (effective)

Package format RTCP REMB tools.ietf.org/html/draft-alvestrand-rmcat-remb-00#section-2.2

The value of the estimated maximum bitrate is encoded in the fields BR Exp and BR Mantissa.

An example implementation can be found in the Chromium code: cs.chromium.org/chromium/src/third_party/webrtc/modules/rtp_rtcp/source/rtcp_packet/remb.cc
In Go, we converted this message to the PacketALFBRemb structure:

When you need to know the bitrate from such a structure, you can do this:

If on the contrary, you have a bitrate and you want to fill in the Exp and Mantissa fields in the PacketALFBRemb structure, then you will do something like this:

Bandwidth and RTCP REMB

This is one of the advantages of the MCU - the ability to make a REMB RTCP package for controlling the outgoing bit rate in the browser:

The MCU sends RTCP REMB to control the outgoing bitrate of the FOO browser.

Also, the MCU can take a bitrate estimate from the recipient and, in accordance with it, change its encoding settings to match the calculated maximum bitrate; it also optimizes the outgoing throughput of the MCU:

MCU changes encoding settings using RTCP REMB values ​​from BAR member

We control bitrate

How can I test the effect of RTCP REMB on the FOO browser bitrate? Pretty easy: just send REMB packets with a variable bit rate (for example, 64Kb and 128Kb every 10 seconds), and then look at the chrome: // webrtc-internals charts .

In Go language, it will be like this:

When you open chrome: // webrtc-internals , you will see something like this:

This is a clear confirmation that you are able to change and monitor the bitrate on your part in real time. Which parameters are best suited to your needs and whether it is necessary to optimize bandwidth and stream quality is up to you :)

You have probably noticed that RTCP REMB is now deprecated . The idea was that the data source (and not the remote receiver / participant) should control the bandwidth - well, in the article we showed that this is still possible and how to do it.

In our case, it was useful to get precise control over the strip and quality :)

Also popular now: