An easy way to connect an arbitrary video source in Qml

Preamble

All of the following is given in the context of Qt version 5.3.1 (as the most relevant at the moment), but it makes sense in the context of any version of the 5.x branch, and possibly even 4.8.x (did not check as unnecessary).

Operating system - Windows, development environment - QtCreator 3.1.2 in conjunction with MinGW and gcc 4.8.2 The essence does not change from the use of other platforms / IDE / compilers.

As the source of the video data, the simplest of the available options was chosen, namely the desktop. Those. The application will display a copy of everything that happens on the desktop. In multi-monitor configurations, as the source, we will use the main screen.

So let's get started

As a starting point, you can read the documentation: “Video Overview: Working with Low Level Video Frames” .

A few points to take from this article:

• The video source must be a descendant of QObject;
• It must declare a property videoSurface of type QAbstractVideoSurface *;
• It should call QAbstractVideoSurface :: start, with the QVideoSurfaceFormat passed, before starting playback;
• It should call QAbstractVideoSurface :: present, with a QVideoFrame passed, to display each frame;
• It should call QAbstractVideoSurface :: stop, when the movie finishes playing.

Writing a code

We are creating a new project “Qt Quick Application”. It is important to choose this type of application type, since in the future we will create a Qml component using C ++.

Next, create a class, a descendant of QObject, and begin to expand it.

As far as the result turned out, it’s quite simple and laconic, I won’t pour a lot of water, but just give the code, with some comments:

DesktopVideoProducer.h:

#pragma once
#include 
#include 
class DesktopVideoProducer : public QObject
{
    Q_OBJECT
public:
    //Для того чтобы класс был доступен для использования в Qml его необходимо регистрировать
    static void registerQmlType();
    explicit DesktopVideoProducer( QObject *parent = 0 );
    ~DesktopVideoProducer();
   //то самое property, упомянутое выше
    Q_PROPERTY( QAbstractVideoSurface* videoSurface READ videoSurface WRITE setVideoSurface )
    QAbstractVideoSurface* videoSurface() const;
    void setVideoSurface( QAbstractVideoSurface* s );
protected:
    void timerEvent( QTimerEvent* );
private:
    void closeSurface();
private:
    QAbstractVideoSurface* _surface;
    QVideoSurfaceFormat _format;
};

DesktopVideoProducer.cpp:

#include "DesktopVideoProducer.h"
#include 
#include 
#include 
#include 
void DesktopVideoProducer::registerQmlType()
{
    //Регистрируем наш класс в составе пакета DesktopVideoProducer,
    //под версией 0.1, под именем DesktopVideoProducer.
    //Нижележащая строчка является подсказкой для парсера типов QtCreator,
    //и она не обязательна.
    // @uri DesktopVideoProducer
    qmlRegisterType(
        "DesktopVideoProducer", 0, 1,
        "DesktopVideoProducer" );
}
DesktopVideoProducer::DesktopVideoProducer( QObject *parent )
    : QObject( parent ), _surface( 0 )
{
    startTimer( 1000 / 15 ); //15 fps
}
DesktopVideoProducer::~DesktopVideoProducer()
{
    closeSurface();
}
QAbstractVideoSurface* DesktopVideoProducer::videoSurface() const
{
    return _surface;
}
void DesktopVideoProducer::setVideoSurface( QAbstractVideoSurface* s )
{
    closeSurface();
    _surface = s;
}
void DesktopVideoProducer::closeSurface()
{
    if( _surface && _surface->isActive() )
        _surface->stop();
}
void DesktopVideoProducer::timerEvent( QTimerEvent* )
{
    if( !_surface )
        return;
    QScreen* screen = QGuiApplication::primaryScreen();
    QDesktopWidget* desktop = QApplication::desktop();
    if( !screen || !desktop )
        return;
    //Получим screenshot и преобразуем в экземпляр класса подходящий для QVideoFrame
    QPixmap screenPixmap = screen->grabWindow( desktop->screen()->winId() );
    QImage screenImage = screenPixmap.toImage();
    QVideoFrame::PixelFormat pixelFormat =
        QVideoFrame::pixelFormatFromImageFormat( screenImage.format() );
    //если формат кадра по какой-то причине поменялся (или это первый кадр)-
    //выполним повторную (первичную) инициализацию surface
    if( screenPixmap.size() != _format.frameSize() ||
        pixelFormat != _format.pixelFormat() )
    {
        closeSurface();
        _format =
            QVideoSurfaceFormat( screenPixmap.size(),
                                 pixelFormat );
        _surface->start( _format );
    }
    //передадим полученный кадр на отрисовку
    _surface->present( QVideoFrame( screenImage ) );
}

main.qml:

import QtQuick 2.2
import QtQuick.Window 2.1
import QtMultimedia 5.0
import DesktopVideoProducer 0.1
Window {
    visible: true
    width: 360
    height: 360
    DesktopVideoProducer {
        id: videoProducer;
    }
    VideoOutput {
        anchors.fill: parent;
        source: videoProducer;
    }
}

main.cpp:

#include 
#include 
#include"DesktopVideoProducer.h"
int main(int argc, char *argv[])
{
    //зарегистрируем DesktopVideoProducer для использования в Qml
    DesktopVideoProducer::registerQmlType();
    //для возможности вызова QApplication::desktop() QGuiApplication недостаточно
    //QGuiApplication app(argc, argv);
    QApplication app(argc, argv);
    QQmlApplicationEngine engine;
    engine.load(QUrl(QStringLiteral("qrc:///main.qml")));
    return app.exec();
}

PS: The full project is available for download from GitHub .