December 26, 2013 at 15:03Miracast standard - old protocols in a new wrapper
Not so long ago (starting with JellyBean 4.2), Google added support for Miracast technology to Android .
The article is devoted to the practical study of this technology by reverse engineering methods.
What is Miracast in a nutshell? This is the next brainchild of the Wi-Fi alliance - the standard for transmitting multimedia content over a Wi-Fi network in peer-to-peer mode. For the user, this means, first of all, that to connect to the TV (for example), he does not need a Wi-Fi router. The two devices, as conceived by the alliance, should communicate directly with each other. This is ensured by using the Wi-Fi Direct standard for the authorship of the same organization. In other words, the new standard solves problems very similar to Apple's AirPlay, Intel WiDi, or the good old DLNA.
Why was it to fence the garden - you ask. Why not take advantage of an existing solution? It will be difficult for me to answer. It is clear that licensing solutions from direct competitors or even from Intel is not a kosher option, which also has a fatal flaw , but why not take the same DLNA, perhaps with a bit of a rasp. Perhaps I wanted something new, with peer-to-peer words that were trendy for a while? I will not guess. One way or another, the technology was implemented in Android, and fresh phones like the Nexus 4 and Samsung Galaxy S3 have it on board.
The situation is worse with TV manufacturers. If DLNA support is already available in almost every modern TV of a sufficiently high level, then things are worse with Miracast. Despite the existence of chips, TV models and projectors that can accept Miracast can be counted on the fingers. However, the situation is likely to change in 2014, but for now - the user can be satisfied with numerous gadgets that receive Wi-Fi signals and convert it to HDMI. Such a thing is stuck in the HDMI-connector of the TV, and now you have a Miracast-enabled device!
One of the engineering samples with the Broadcom chip fell into my tenacious hands:
After making sure that everything worked with an Android-smartphone with a bang, I thought about the question - is it possible to establish broadcasting through Miracast directly from under Linux? After all, what is Android inside? The same Linux ...
For starters, I wanted to understand how the Miracast protocol stack looks like? What is behind a beautiful name? Whether the video signal is chased directly in Ethernet frames or is IP and even higher-level protocols used. Unfortunately, the standard itself , although open, is far from free, so I had to find other, more traditional ways of research. In some presentation, I grabbed the keywords - MPEG-TS and RTSP, and this made it possible to untwist the tangle further. If I know anything at all, then RTSPIs TCP, and TCP is IP. And IP is the right protocol to listen to with tcpdump! No sooner said than done, running tcpdump on the Nexus and turning on the Wireless display in the settings, after 5 minutes I had a packet dump acceptable for further analysis.
Temporarily postponing the difficulties of connecting via Wi-Fi, I immediately took up the analysis of the TCP stream. And here is what I saw:
Not true, it resembles a regular RTSP. So part of the job is done. It remains to be understood how the Miracast-based RTSP implementation differs from the standard one. For those who have never encountered RTSP (Real Time Streaming Protocol), let me remind you that it is used to control multimedia stream from the server on the client. In other words, it allows to issue such commands as PLAY, PAUSE, TEARDOWN, etc. It is also possible to exchange options and adjust settings. It was {GET | SET} _PARAMETER that became my main headache in the analysis. Having no standard at hand, I could not know what all these wfd_video_formats, wfd_audio_codecs, etc. mean. But I could guess!
Since I understood from the analysis of MPEG-TS frames that the standard resolution was 720x480, and the H.264 (AVC) codec, it was a good idea to create a video file with exactly the same parameters, and then fields like wfd_video_formats can be left unchanged! Rummaging through DVDs, I transcoded a small VOB from the Cracker television series to the format I needed using ffmpeg. Now all that remained was to feed the file to the server. But for this you need to find a server!
In order not to write an RTSP server myself (which was not part of my plans), I started looking at Open Source options that would be easy to modify to a state compatible with Miracast. If you carefully looked at the logs from tcpdump, you might notice a few oddities. The traditional client-server RTSP model has been replaced by peer-to-peer interaction. This means that the activity in the requests can come not only from the client (in this case the TV or the projector acts), but also from the “server” (that is, the phone or computer). Why it was necessary to do this is not clear, but the fact remains - both the “client” and the “server” can send requests when they feel like, which negates their traditional roles. Nevertheless, I will continue to refer to the side that sends the video signal as the server (in our case, Linix-PC), and the side
So, after several hours of searching, I settled on live555 . This server is written in C ++, distributed under the LGPL license and supports both RTSP and broadcasting in MPEG-TS. Looking at the RTSP handler, I realized that it is quite possible to process it under the peer-to-peer specifics of Miracast. But, it remained to force the client (i.e. the Miracast gadget) to connect to Linux!
This task was more difficult than Goethe's Faust. Before, I had never set up even regular Wi-Fi in Linux, rightly believing that the wires were somehow more reliable. What can we say about Wi-Fi Direct. However, after reading a stack of manuals, I realized that I had to dig in the direction of the mysterious WPA supplicant. What is this supplicant for? It provides authentication when connecting via Wi-Fi to an access point or to another node. As I wrote above, Miracast works in p2p mode, i.e. devices communicate directly bypassing routers. This feature is fortunately supported in recent versions of wpa_supplicant. I don’t know exactly when p2p support was added, but it is already in version 2.1-devel.
However, updating supplicant is not enough! You must still have configuration files for it. With a sin in half, I wrote a configuration acceptable for my device (NetGear, WNA1100 Wireless-N 150 [Atheros AR9271]), maybe it will suit you as well.
So, in the /etc/wpa_p2p.conf file we write:
Next, you need a shell script to run supplicant:
That's all (I’ll clarify that this configuration works in the Ubuntu-based Linux Mint 13 Maya distribution, the kernel version is 3.2.0-57-generic).
Next, you need to master such a utility as wpa_cli, it is it that allows you to control the connection "manually".
After running wpa_supplicant through a script, you need to open a separate console and issue something like:
This is the command interface for supplicant. Turning on the gadget, we can use the p2p_find command to find all the devices in the district that are ready to connect to us in p2p mode. Next, using the p2p_connect command, we make the connection itself.
Here is an example log for my device:
In principle, everything is clear from the log, except perhaps for the mysterious word 'pbc' in the p2p_connect command after the device address. What does it mean? This is one of the authentication options when connecting via Wi-Fi direct. It means - Push Button Control. This is a simplified authentication that does not require the user to enter a password or even a pin code. Just at the time of connection you need to press a button on the device, and authentication will be considered successful.
So, from the log we see that the connection has been successful. And now we are able to get the IP address for the wlan0 interface.
The DHCP server in this case will be the TV or projector. We introduce in a separate terminal:
If we then run tcpdump, we will find attempts to send a SYN packet to port 7236. This port is different from the standard port for RTSP (554), but this should not scare us. Most importantly, the gadget wants to negotiate with us! By launching the already slightly modified livemedia server on this port (7236), we get the opportunity to debug the actual “client-server” interaction.
I will not bore the reader with the details of debugging the protocol, I will only say that all the problems have been resolved in one way or another. And finally, the result is obvious - I was able to watch video from my PC through the new-fangled Miracast!
Do you need it? I do not know. In any case, understanding the new standard is always interesting (unless of course it is not ASN.1).
For those who were too lazy to delve into the technical details, I will briefly outline the connection procedure for Miracast-based devices:
Of the obvious flaws of Miracast (not mentioned in the Wiki), I would note the following:
That's all. If you have any questions, ask in the comments.
The article is devoted to the practical study of this technology by reverse engineering methods.
What is Miracast in a nutshell? This is the next brainchild of the Wi-Fi alliance - the standard for transmitting multimedia content over a Wi-Fi network in peer-to-peer mode. For the user, this means, first of all, that to connect to the TV (for example), he does not need a Wi-Fi router. The two devices, as conceived by the alliance, should communicate directly with each other. This is ensured by using the Wi-Fi Direct standard for the authorship of the same organization. In other words, the new standard solves problems very similar to Apple's AirPlay, Intel WiDi, or the good old DLNA.
Why was it to fence the garden - you ask. Why not take advantage of an existing solution? It will be difficult for me to answer. It is clear that licensing solutions from direct competitors or even from Intel is not a kosher option, which also has a fatal flaw , but why not take the same DLNA, perhaps with a bit of a rasp. Perhaps I wanted something new, with peer-to-peer words that were trendy for a while? I will not guess. One way or another, the technology was implemented in Android, and fresh phones like the Nexus 4 and Samsung Galaxy S3 have it on board.
The situation is worse with TV manufacturers. If DLNA support is already available in almost every modern TV of a sufficiently high level, then things are worse with Miracast. Despite the existence of chips, TV models and projectors that can accept Miracast can be counted on the fingers. However, the situation is likely to change in 2014, but for now - the user can be satisfied with numerous gadgets that receive Wi-Fi signals and convert it to HDMI. Such a thing is stuck in the HDMI-connector of the TV, and now you have a Miracast-enabled device!
One of the engineering samples with the Broadcom chip fell into my tenacious hands:
After making sure that everything worked with an Android-smartphone with a bang, I thought about the question - is it possible to establish broadcasting through Miracast directly from under Linux? After all, what is Android inside? The same Linux ...
For starters, I wanted to understand how the Miracast protocol stack looks like? What is behind a beautiful name? Whether the video signal is chased directly in Ethernet frames or is IP and even higher-level protocols used. Unfortunately, the standard itself , although open, is far from free, so I had to find other, more traditional ways of research. In some presentation, I grabbed the keywords - MPEG-TS and RTSP, and this made it possible to untwist the tangle further. If I know anything at all, then RTSPIs TCP, and TCP is IP. And IP is the right protocol to listen to with tcpdump! No sooner said than done, running tcpdump on the Nexus and turning on the Wireless display in the settings, after 5 minutes I had a packet dump acceptable for further analysis.
Temporarily postponing the difficulties of connecting via Wi-Fi, I immediately took up the analysis of the TCP stream. And here is what I saw:
OPTIONS * RTSP / 1.0 Date: Fri, 08 Mar 2013 12:37:54 +0000 Server: Mine / 1.0 CSeq: 1 Require: org.wfa.wfd1.0 RTSP / 1.0 200 OK CSeq: 1 Public: org.wfa.wfd1.0, GET_PARAMETER, SET_PARAMETER OPTIONS * RTSP / 1.0 CSeq: 1 Require: org.wfa.wfd1.0 RTSP / 1.0 200 OK Date: Fri, 08 Mar 2013 12:37:54 +0000 Server: Mine / 1.0 CSeq: 1 Public: org.wfa.wfd1.0, SETUP, TEARDOWN, PLAY, PAUSE, GET_PARAMETER, SET_PARAMETER GET_PARAMETER rtsp: //localhost/wfd1.0 RTSP / 1.0 Date: Fri, 08 Mar 2013 12:37:54 +0000 Server: Mine / 1.0 CSeq: 2 Content-Type: text / parameters Content-Length: 83 wfd_content_protection wfd_video_formats wfd_audio_codecs wfd_client_rtp_ports RTSP / 1.0 200 OK CSeq: 2 Content-Type: text / parameters Content-Length: 751 wfd_content_protection: none wfd_video_formats: 00 00 02 10 0001bdeb 3fffffff 00000fff 00 0000 0000 11 none none, 02 08 0001bdeb 3fffffff 00000fff 00 0000 0000 11 none none, 02 04 0001bdeb 3fffffff 00000fff 00 0000 0000 11 none none, 02 02 0001bdeb 3fffffff 00000ff none, 02 01 0001bdeb 3fffffff 00000fff 00 0000 0000 11 none none, 01 10 0001bdeb 3fffffff 00000fff 00 0000 0000 11 none none, 01 08 0001bdeb 3fffffff 00000fff 00 0000 0000 11 none none, 01 04 0001bdeb 3fffffff 00000fff 00 0000 0000 11 none none, 01 02 0001bdeb 3fffffff 00000fff 00 0000 0000 11 none none, 01 01 0001bdeb 3fffffff 00000fff 00 0000 0000 11 none none wfd_audio_codecs: LPCM 00000003 00 wfd_client_rtp_ports: RTP / AVP / UDP; unicast 6500 0 mode = play SET_PARAMETER rtsp: //localhost/wfd1.0 RTSP / 1.0 Date: Fri, 08 Mar 2013 12:37:54 +0000 Server: Mine / 1.0 CSeq: 3 Content-Type: text / parameters Content-Length: 248 wfd_video_formats: 28 00 02 02 00000020 00000000 00000000 00 0000 0000 00 none none wfd_audio_codecs: LPCM 00000002 00 wfd_presentation_URL: rtsp: //192.168.16.40/wfd1.0/streamid=0 none wfd_client_rtp_ports: RTP / AVP / UDP; unicast 6500 0 mode = play RTSP / 1.0 200 OK CSeq: 3 SET_PARAMETER rtsp: //localhost/wfd1.0 RTSP / 1.0 Date: Fri, 08 Mar 2013 12:37:54 +0000 Server: Mine / 1.0 CSeq: 4 Content-Type: text / parameters Content-Length: 27 wfd_trigger_method: SETUP RTSP / 1.0 200 OK CSeq: 4 SETUP rtsp: //192.168.16.40/wfd1.0/streamid=0 RTSP / 1.0 CSeq: 2 Transport: RTP / AVP / UDP; unicast; client_port = 6500 RTSP / 1.0 200 OK Date: Fri, 08 Mar 2013 12:37:55 +0000 Server: Mine / 1.0 CSeq: 2 Session: 1219569791; timeout = 30 Transport: RTP / AVP / UDP; unicast; client_port = 6500; server_port = 15550 PLAY rtsp: //192.168.16.40/wfd1.0/streamid=0 RTSP / 1.0 CSeq: 3 Session: 1219569791 RTSP / 1.0 200 OK Date: Fri, 08 Mar 2013 12:37:55 +0000 Server: Mine / 1.0 CSeq: 3 Session: 1219569791; timeout = 30 Range: npt = now- SET_PARAMETER rtsp: //localhost/wfd1.0 RTSP / 1.0 Date: Fri, 08 Mar 2013 12:38:07 +0000 Server: Mine / 1.0 CSeq: 5 Content-Type: text / parameters Content-Length: 30 wfd_trigger_method: TEARDOWN RTSP / 1.0 200 OK CSeq: 5 TEARDOWN rtsp: //192.168.16.40/wfd1.0/streamid=0 RTSP / 1.0 CSeq: 4 Session: 1219569791 RTSP / 1.0 200 OK Date: Fri, 08 Mar 2013 12:38:09 +0000 Server: Mine / 1.0 CSeq: 4 Session: 1219569791; timeout = 30 Connection: close
Not true, it resembles a regular RTSP. So part of the job is done. It remains to be understood how the Miracast-based RTSP implementation differs from the standard one. For those who have never encountered RTSP (Real Time Streaming Protocol), let me remind you that it is used to control multimedia stream from the server on the client. In other words, it allows to issue such commands as PLAY, PAUSE, TEARDOWN, etc. It is also possible to exchange options and adjust settings. It was {GET | SET} _PARAMETER that became my main headache in the analysis. Having no standard at hand, I could not know what all these wfd_video_formats, wfd_audio_codecs, etc. mean. But I could guess!
Since I understood from the analysis of MPEG-TS frames that the standard resolution was 720x480, and the H.264 (AVC) codec, it was a good idea to create a video file with exactly the same parameters, and then fields like wfd_video_formats can be left unchanged! Rummaging through DVDs, I transcoded a small VOB from the Cracker television series to the format I needed using ffmpeg. Now all that remained was to feed the file to the server. But for this you need to find a server!
In order not to write an RTSP server myself (which was not part of my plans), I started looking at Open Source options that would be easy to modify to a state compatible with Miracast. If you carefully looked at the logs from tcpdump, you might notice a few oddities. The traditional client-server RTSP model has been replaced by peer-to-peer interaction. This means that the activity in the requests can come not only from the client (in this case the TV or the projector acts), but also from the “server” (that is, the phone or computer). Why it was necessary to do this is not clear, but the fact remains - both the “client” and the “server” can send requests when they feel like, which negates their traditional roles. Nevertheless, I will continue to refer to the side that sends the video signal as the server (in our case, Linix-PC), and the side
So, after several hours of searching, I settled on live555 . This server is written in C ++, distributed under the LGPL license and supports both RTSP and broadcasting in MPEG-TS. Looking at the RTSP handler, I realized that it is quite possible to process it under the peer-to-peer specifics of Miracast. But, it remained to force the client (i.e. the Miracast gadget) to connect to Linux!
This task was more difficult than Goethe's Faust. Before, I had never set up even regular Wi-Fi in Linux, rightly believing that the wires were somehow more reliable. What can we say about Wi-Fi Direct. However, after reading a stack of manuals, I realized that I had to dig in the direction of the mysterious WPA supplicant. What is this supplicant for? It provides authentication when connecting via Wi-Fi to an access point or to another node. As I wrote above, Miracast works in p2p mode, i.e. devices communicate directly bypassing routers. This feature is fortunately supported in recent versions of wpa_supplicant. I don’t know exactly when p2p support was added, but it is already in version 2.1-devel.
However, updating supplicant is not enough! You must still have configuration files for it. With a sin in half, I wrote a configuration acceptable for my device (NetGear, WNA1100 Wireless-N 150 [Atheros AR9271]), maybe it will suit you as well.
So, in the /etc/wpa_p2p.conf file we write:
ctrl_interface=/var/run/wpa_supplicant
ap_scan=1
device_name=JellyFish
device_type=1-0050F204-1
Next, you need a shell script to run supplicant:
sudo iwconfig wlan0 mode ad-hoc
sudo ip link set wlan0 up
sudo wpa_supplicant -Dnl80211 -c /etc/wpa_p2p.conf -i wlan0 -dt
That's all (I’ll clarify that this configuration works in the Ubuntu-based Linux Mint 13 Maya distribution, the kernel version is 3.2.0-57-generic).
Next, you need to master such a utility as wpa_cli, it is it that allows you to control the connection "manually".
After running wpa_supplicant through a script, you need to open a separate console and issue something like:
sudo wpa_cli
This is the command interface for supplicant. Turning on the gadget, we can use the p2p_find command to find all the devices in the district that are ready to connect to us in p2p mode. Next, using the p2p_connect command, we make the connection itself.
Here is an example log for my device:
wpa_cli v2.1-devel Selected interface 'wlan0' Interactive mode > p2p_find Ok <3> P2P-DEVICE-FOUND 02: 90: 4c: 04: 04: 04 p2p_dev_addr = 02: 90: 4c: 04: 04: 04 pri_dev_type = 7-0050F204-1 name = 'MLT-52-2123' config_methods = 0x4688 dev_capab = 0x25 group_capab = 0xa > > p2p_connect 02: 90: 4c: 04: 04: 04 pbc Ok <3> P2P-FIND-STOPPED <--- Here you need to press the button on the device <3> P2P-GO-NEG-SUCCESS <4> Failed to initiate AP scan <4> Failed to initiate AP scan <4> Failed to initiate AP scan <4> Failed to initiate AP scan <3> CTRL-EVENT-SCAN-RESULTS <3> WPS-AP-AVAILABLE-PBC <3> SME: Trying to authenticate with 02: 90: 4c: 04: 84: 04 (SSID = 'DIRECT-fCMLT-52-2123' freq = 2412 MHz) <3> Trying to associate with 02: 90: 4c: 04: 84: 04 (SSID = 'DIRECT-fCMLT-52-2123' freq = 2412 MHz) <3> CTRL-EVENT-SCAN-RESULTS <3> WPS-AP-AVAILABLE-PBC <3> SME: Trying to authenticate with 02: 90: 4c: 04: 84: 04 (SSID = 'DIRECT-fCMLT-52-2123' freq = 2412 MHz) <3> Trying to associate with 02: 90: 4c: 04: 84: 04 (SSID = 'DIRECT-fCMLT-52-2123' freq = 2412 MHz) <3> Associated with 02: 90: 4c: 04: 84: 04 <3> CTRL-EVENT-EAP-STARTED EAP authentication started <3> CTRL-EVENT-EAP-PROPOSED-METHOD vendor = 14122 method = 1 <3> CTRL-EVENT-EAP-METHOD EAP vendor 14122 method 1 (WSC) selected <3> WPS-CRED-RECEIVED <3> WPS-SUCCESS <3> P2P-GROUP-FORMATION-SUCCESS <3> CTRL-EVENT-EAP-FAILURE EAP authentication failed <3> CTRL-EVENT-DISCONNECTED bssid = 02: 90: 4c: 04: 84: 04 reason = 3 locally_generated = 1 <3> CTRL-EVENT-SCAN-RESULTS <3> WPS-AP-AVAILABLE <3> SME: Trying to authenticate with 02: 90: 4c: 04: 84: 04 (SSID = 'DIRECT-fCMLT-52-2123' freq = 2412 MHz) <3> Trying to associate with 02: 90: 4c: 04: 84: 04 (SSID = 'DIRECT-fCMLT-52-2123' freq = 2412 MHz) <3> Associated with 02: 90: 4c: 04: 84: 04 <3> WPA: Key negotiation completed with 02: 90: 4c: 04: 84: 04 [PTK = CCMP GTK = CCMP] <3> CTRL-EVENT-CONNECTED - Connection to 02: 90: 4c: 04: 84: 04 completed [id = 0 id_str =] <3> P2P-GROUP-STARTED wlan0 client ssid = "DIRECT-fCMLT-52-2123" freq = 2412 psk = fd435c6683ae5d7c9e3398dab15cc1b80d7f308b3fe7330db044ea90dcf7ac31 go_dev_addr 04IST: 04IST: 04IST: 04IST = 04IST = 02IST = 04IST = 02IST = 04IST = 02IST = 04IST = 02IST = 04IST = 04IST = 02: 04: 04IST = 04IST = 02IST = 04IST = 2
In principle, everything is clear from the log, except perhaps for the mysterious word 'pbc' in the p2p_connect command after the device address. What does it mean? This is one of the authentication options when connecting via Wi-Fi direct. It means - Push Button Control. This is a simplified authentication that does not require the user to enter a password or even a pin code. Just at the time of connection you need to press a button on the device, and authentication will be considered successful.
So, from the log we see that the connection has been successful. And now we are able to get the IP address for the wlan0 interface.
The DHCP server in this case will be the TV or projector. We introduce in a separate terminal:
sudo dhclient wlan0
If we then run tcpdump, we will find attempts to send a SYN packet to port 7236. This port is different from the standard port for RTSP (554), but this should not scare us. Most importantly, the gadget wants to negotiate with us! By launching the already slightly modified livemedia server on this port (7236), we get the opportunity to debug the actual “client-server” interaction.
I will not bore the reader with the details of debugging the protocol, I will only say that all the problems have been resolved in one way or another. And finally, the result is obvious - I was able to watch video from my PC through the new-fangled Miracast!
Do you need it? I do not know. In any case, understanding the new standard is always interesting (unless of course it is not ASN.1).
For those who were too lazy to delve into the technical details, I will briefly outline the connection procedure for Miracast-based devices:
- Using Wi-Fi direct, devices find each other (usually - the video source finds the display device)
- Using this or that form of authentication (in our case, pbc), devices are combined into a P2P group
- One of the devices receives an IP address via DHCP (in our case, this is a source of video data)
- An RTSP server starts on a data source on port 7236
- The client connects to the RTSP server and requests a predefined URL (/wfd1.0/streamid=0)
- The RTSP server starts transmitting video (and possibly audio) data in the form of MPEG-TS packed in RTP packets.
- The client unpacks the data and displays it on the output device.
Of the obvious flaws of Miracast (not mentioned in the Wiki), I would note the following:
- If you connect to a Miracast device, you lose the ability to work through normal (not P2P) Wi-Fi. To use traditional Wi-Fi and Wi-Fi direct at the same time, you need a special dual-channel Wi-Fi adapter. It is not available on all phones!
- The picture quality on dynamic scenes suffers even at a resolution of 720x480, 30 FPS. I'm not talking about Full HD. Of course, with the advent of more powerful processors, the picture will change, but so far everything is sad.
That's all. If you have any questions, ask in the comments.