Record video from a Mac OS X screen using open-source

Sometimes you need to record a demo of the program, but there are no suitable tools at hand. Moreover, free utilities for these purposes cannot be found at all, and paid ones are not a fact that they can do it right. As corrected me in the comments, the regular version of QuickTime Player writes great video. Further it is recommended to read only to fans of open-source and unusual solutions.
In my case, there was a need to record the game on a simulator iPhone and Android. The programmer inside me immediately suggested writing a bunch of code, both for iOS / Android, and for Mac itself, dumping frames via OpenGL, etc. Having stopped these urges, I decided to find ready-made solutions, and then make an article here as a memo.

Prelude

A search of the forums showed that people with similar tasks fall into two categories: those who buy specialized programs with the ability to capture (QuickTime Pro, etc.) and those who use hacked / trial / beta versions of the same programs. After digging a little deeper, I found out that in the Linux / Unix world there is no such problem at all since the x11grab module was added to ffmpeg. This module intercepts the buffer of X applications and then through ffmpeg the video is compressed into an intermediate format (qtrle, raw, x264 lossless, etc.). Mac also has an X server, but only programs ported from Linux but not adapted for Cocoa output information through it. This was the starting point of my experiments.

Tools: ffmpeg + x11grab + Xvfb

In MacPorts in ffmpeg, the x11grab module is basically missing. It didn't work out the whole time from the source code the first time, so I decided to patch the port file for ffmpeg-devel:
/opt/local/var/macports/sources/rsync.macports.org/release/tarballs/ports/multimedia/ffmpeg-devel/Portfile
I added the lines --enable-x11grab and --enable-shared to the nonfree configuration to activate x11grab.
Looking ahead, I’ll say that there was a flaw in this method, because I don’t upload Portfile with edits, but describe it here only for reference.

When the X program is displayed in session (display): 0.0 by default, video recording gives a small frame rate, mainly because of the size of the screen (iMac has 2560x1440), and most likely also because of the graphics rendering, so I decided to redirect programs in a virtual display with a small resolution. This is done through the Xvfb project., which is installed without problems from the ports.
It starts quite simply through the terminal (size with a margin):
   Xvfb :1 -screen 0 1024x800x15 -shmem

It also just connects to this virtual display ffmpeg:
   ffmpeg -r 30 -s 1024x768 -f x11grab -i :1.0+0,20 -vcodec qtrle target.mov

At this stage, I was waiting for an error with Shared Memory overflow, which for some reason in OS X was set to an indecent value of 4mb. A temporary increase in its size is described in Apple’s recommendations for servers and other sources :
   sudo sysctl -w kern.sysv.shmmax=67108864
   sudo sysctl -w kern.sysv.shmall=67108864

VirtualBox Session Recording

The next step, I decided to output something useful through the X server. The first thought was to build an X-version of VirtualBox, and already show literally anything there, but VirtualBox for poppies has long migrated to Cocoa, because it was a dead end. The second common thought was to connect via RDP to the virtual machine and record the rdesktop session, since it works under X. Activating RDP under VirtualBox is quite simple, but requires the installation of Extension Pack from the official site .
Connecting rdesktop with screen output: 0.1
   DISPLAY=:1.0 rdesktop -xl localhost

After these steps, ffmpeg starts quite successfully writing video to a .mov file. In my case, it was successfully turned up on the arm of Android x86:


Unfortunately, the video turns out to be rather twitchy, rdp compression affects it, so the animation is not so good to shoot like that.

The next step, I decided to switch from RDP to VNC. VirtualBox has a VNC server built in, but not in public builds, but in compiled from ports or source codes. I did not have to do any port manipulations, after building the virtualbox port, I got version 4.1.14, which is quite possible to work with.

The only unpleasant moment was that VNC is not accessible through the interface, but only when launched in headless mode:
   VBoxHeadless -startvm 'Android x86' -v on --vnc
You have to manage this mode either through the second VNC session or again through RDP, which is not very convenient, but generally tolerable. To capture the VNC stream, vncviewer was used, redirected to the virtual X display:
   vncviewer localhost -ViewOnly -display :1.0 -PreferredEncoding raw -FullColor

The result of all these frills was a 5-minute video with honest 30 fps in a resolution of 1024x768:

(I apologize in advance for the quality of the content, I'm still not a professional player)

If you look closely, sometimes there are noticeable pauses for a few seconds. Unfortunately, this problem could not be defeated, and the approach itself turned out to be rather cumbersome. In general, this is enough for the simplest demonstration of the game in Android, so I switched to the next task - shooting a video simulator of the iPhone.

Full Screen VNC Capture

OS X has built-in remote access that works simultaneously under two protocols - ARD and VNC. Prior to the release of Lion 10.7, it was possible to enable Screen Sharing in the system settings and connect to the current session with any VNC client. Starting with 10.7, serious changes began: all types of compression were thrown except ZRLE, not all clients can connect, and after connecting we see a gray login screen, and only after entering the user password we connect to the session. This is great for administrators, but for my task, on the contrary, it created only obstacles. The vncviewer program (aka RealVNC) in the latest versions already knows how to connect to OS X, but does not know how to enter the user password, therefore this path also turned out to be deadlock.
As an alternative, I took a free VNC server from TestPlant (from osxvnc and also Vine). Version 3.0 with sourceforge is deprecated, so you need to build a new one from the source or take it from the TestPlant website .

An insignificant bug of this server is that from time to time the client disappears with the error "unknown message type 131". It is treated by restarting the server.

With the already established bunch of ffmpeg + x11grab + Xvfb and vncviewer, it turned out to shoot a full-screen video of the current OS X session where the iPhone simulator was running:


The size of the virtual buffer I chose was obviously lower than the screen resolution in order to shoot the upper left corner. The result came out quite worthy, but unfortunately, with a low FPS - the animation in the game frankly slowed down. Moreover, the pauses, which were quite rare with the schematic from VirtualBox, became much more pronounced.
At this stage, I conducted several experiments, reassembled the VNC server and client from source, set the minimum screen resolution, but still did not achieve a good result. After a few hours, it became clear that the server itself was sending frames with some delay. After digging in the code, it turned out that the server intentionally pauses between screen updates:

        /* OK, now, to save bandwidth, wait a little while for more updates to come along. */
        /* REDSTONE - Lets send it right away if no rfbDeferUpdateTime */
        if (rfbDeferUpdateTime > 0 && !cl->immediateUpdate && !cl->needNewScreenSize) {
            pthread_mutex_unlock(&cl->updateMutex);
            usleep(rfbDeferUpdateTime * 1000);
            pthread_mutex_lock(&cl->updateMutex);
        }

It turned out that the rfbDeferUpdateTime variable has an initial value of 40 ms, but it is quite controllable and is set through the command line. In Vine Server itself, there is a separate field for this:

I set the value with a margin of 15, which gives a maximum frame rate of 66 fps. After that, the lags stopped, but there were noticeable pauses in the video. Theoretically, from such a video sequence they can be cut and assembled something acceptable, but I wanted a more universal solution.

Ace up your sleeve: vnc2flv

Now I had a video stream in vnc format of excellent quality, it remained only to write it to a file. I again suppressed the original programming desire to write my own dumper and found the vnc2swf project in the bowels of the Internet , and then its successor vnc2flv . The skeptical attitude to the grabber on Python passed immediately after the first results - the program records the video stream in lossless quality and WQHD resolution with 15+ fps! I start it without distortions from Xvfb, directly connecting to the VNC server:
   flvrec.py -r 30 127.0.0.1

To increase fps, it is enough to reduce the resolution to 1280x720. Interestingly, you can restart Vine VNC, it will capture the current screen resolution, and then you can safely switch to the native resolution and enable recording.
Installing vnc2flv is very simple and described on the site, there are no special pitfalls here.
The finished video can be processed in your favorite video editor, trim the excess and convert to the desired format. I use VirtualDub running under wine, but this is already a matter of habit. Here is the result of this whole epic:




The video is quite clear, without jerks and lags. Animation is recorded normally. As for me, it is quite possible to use literally anything from the Mac OS screen to record. Literally just the cursor is missing, but you can survive it anyway.

Afterword: Dead End Branches

I will list here the dead ends and rakes that I met, in case someone searches for their own method.

VLC Player - starting with OS X 10.7 it can’t record video from the
CaptureMe screen - crashes, writes an empty
SIMBL video plugin for iPhone Simulator - requires modification of the simulator version in the code and rebuilding, doesn’t write iPad video, generally
osxvnc 3.0 didn’t work for me c SourceForge - crash on 10.7
RealVNC Server - requires a
VirtualBox license key , video recording mode - died 3 years ago,
ffmpeg + x11grab is not supported and does not work - it is promising and flexible, but gives incurable pauses in the video. Here is a link to the blog from which I took this idea.
Apple HTTP Live Streaming - no streaming tools