QtQuick / QML as a gaming UI

    The article has been updated with useful comments. Many thanks to all commentators for important clarifications and additions.

    In games with a complex UI, creating your own library for displaying it and tools for easy editing can be a very long and difficult task. I really want to solve it once and for all, and not do it again in every new project, and even in every new company.

    The solution is to use ready-made universal UI libraries. Their current generation is represented by such "monsters" as Scaleform and Coherent UI , although if you really want to write UI in HTML, you can just take Awesomium .

    Unfortunately, this trinity, with all its advantages, has one significant drawback - terrible brakes, especially on mobile devices (several years ago, I personally observed how a practically empty screen on Scaleform consumed 50% of the frame time on iPhone4).

    Against this background, I was always wondering why no one uses Qt in games - a library that has worked well in desktop applications. In fact, this statement is not entirely true - the Qt project has a list of games in the wiki , but it has almost no modern professional projects.

    However, the reason why the familiar old Qt Widgetsthey are not used in games, it lies on the surface: they are not designed to be used in conjunction with OpenGL or DirectX rendering. Attempts to cross them give pretty poor performance even on the desktop , and there’s nothing to say about cell phones.

    However, for quite some time, Qt has a much more suitable library for this task: QtQuick . Its default controls render faster, and the ability to specify a description of the UI in text format is great for quickly setting up and changing the look of the game.

    However, I still have not heard about the use of Qt in professional game dev. There were no articles on the topic either, so I decided to figure it out myself - either everyone knows something that I don’t know (but don’t tell!), Or simply don’t see a good opportunity to save on development time.

    Arguments against:



    I'll start with the thing farthest from technical issues, namely with licensing . Qt uses a dual license - LGPL3 and commercial. This means that if you are interested, including, in platforms where dynamic linking is not possible (iOS), you will have to fork out $ 79 per month for each employee who uses Qt. “To use”, as I understand it, is at least just to assemble a project with libraries, that is, you will have to pay for each programmer on the project.

    ChALkeRx clarifies that in order to use static linking it is not necessary to buy a commercial license or lay out the source code - just lay out the object files, a set of which will allow you to rebuild your application with the new static version of Qt.

    The money is not very big, but still not for free. And there is another very interesting point: it is advisable to obtain a commercial Qt license as soon as you start using Qt in your project. Otherwise, when you try to get a license you will be asked to "contact our specialists to discuss the conditions." They are understandable: not only in our country, smart citizens would have guessed for five years to use the free version for the entire development, and only to buy a license for 1 month to build the final build!

    Perhaps the most important technical argument against Qt is its weight. Almost empty desktop application using QML takes up more than 40Mb (with dynamic linking of DLL). On Android, the sizes will be slightly smaller, on the order of 25Mb (in the expanded form, the APK will be noticeably easier), but for a mobile platform it’s just VERY much! Qt offer a crutch that allows you to install libraries on a user's phone once, and use them from different applications ( Ministro ), but this crutch is obviously available only on Android, and we would like to somehow solve the issue with sizes on iOS and Windows Phone ...

    However, lamenting over the overweight libraries, you should not forget that the competitors - the above-mentioned Scaleform and Coherent - are not much better in this regard, both give out empty applications of tens of megabytes in size. Unity - a bit lighter, but still, around 10Mb. Therefore, here Qt loses a lot only to its own, task-optimized developments.

    In conclusion, I will mention one more potential drawback - Qt is not ready for use under the Web (Emscripten). For most developers, this is not very important, but for example, we are engaged in this area, and here it is not possible to use Qt, although work is ongoing in this area .

    Arguments for:



    The main argument for using QtQuick / QML is a convenient UI description format, as well as a visual editor for it. Plus, a large ready-made set of controls.

    It is worth mentioning the ability to write some part of the UI code in JavaScript inside QML, for example, any simple arithmetic that relates the state of fields of different objects - an opportunity that is very rarely available in makeshift UI libraries (and is often necessary).

    However, it is worth noting that Qt Designer is not a Visual Studio form designer. Even for the basic controls that come with Qt, it does not allow editing all their possible properties (for example, because they can be added dynamically). In particular, you cannot assign images to a button through an editorfor pressed and released position. And this is only the beginning of the problems. On the other hand, combining the use of a visual and text editor, all these problems can be overcome. You just don’t have to rely on what Qt Designer can give to the artist, and he will set everything up with the mouse without getting into the textual representation.

    Performance, in my opinion, QtQuick is acceptable. In the latest release of Qt 5.7, they promised to further noticeably improve it with the new QtQuick Controls 2.0, sharpened for mobile platforms.

    Technical features



    Now let's move on to the most interesting - the technical features of using Qt in the game.

    Main cycle



    The first thing to face is that Qt prefers to be the master of the main loop. At the same time, many game engines also claim this. Someone has to give in. In my case, the Nya engine that we use at work parted with the main loop without problems, and, after minimal initialization, it easily uses the OpenGL context created by Qt. But even if your engine refuses to release the main cycle from tenacious paws, then this is not the end of the world. It is enough in your loop to call the processEvents method on the Qt class of the application. An example implementation is provided on StackOverflow , along with criticism.

    DmitrySokolovindicates that there are at least two more ways to make friends with the Qt renderer and your engine: firstly, you can render your scene into a texture that will be drawn as one of the components of the QtQuick stage graph, as described in Mixing Scene Graph and OpenGL . Secondly, you can use the QQuickRenderControl object . Regarding the latter, there is a useful article on Habré , which, in particular, demonstrates the possibility of using two (shared) contexts for Qt and rendering the game so as not to bother so much with states.

    If you went by transferring the main loop into Qt's hands, the question arises - when will we render our game? The QQuickView object into which the UI is loaded for display provides beforeRendering signalsand afterRendering , which you can subscribe to. The first will work before rendering the UI - it's time to render most of the game scene. The second - after the UI is drawn, and here you can draw some more beautiful little particles, well, or suddenly some models that are supposed to be on top of the UI (say, a 3D character doll in the equipment window). IMPORTANT! When connecting signals, specify the connection type Qt :: ConnectionType :: DirectConnection , otherwise you will receive an error due to an attempt to access the OpenGL context from another stream.

    At the same time, one must not forget to prohibit Qt from clearing the screen before drawing the UI - otherwise all our works will be erased (setClearBeforeRendering (false) ).

    Also, in afterRendering it makes sense to call the QQuickView function update. The fact is that Qt usually saves our time and money, and until nothing has changed in it itself, it will not redraw the UI, and as a result, it will not call these same before / afterRendering, and we won’t be able to draw anything either. Calling update will cause the next frame to draw everything again. If you want to limit the number of frames per second, then you can immediately sleep.

    Something else about rendering



    We need to remember that we have a common OpenGL context with Qt. This means that you need to handle it carefully. Firstly, Qt will do what he wants with it. Therefore, when we need to draw something ourselves (in before or afterRendering), then firstly, we need to make this context current ( m_qt_wnd-> openglContext () -> makeCurrent (m_qt_wnd) ), and secondly, set he needs all the settings we need. In the Nya engine, this is done with a single call to apply_state (true), but in your engine it can be more complicated.

    Secondly, after we drew our own , we need to return the context to the acceptable Qt state by calling m_qt_wnd-> resetOpenGLState ();

    By the way, it is worth considering that since the OpenGL context creates Qt, and not your engine, you must make sure that your engine does not do anything extra before the context is created. To do this, you can subscribe to the openglContextCreated signal , well, or do the initialization in the first call beforeRendering.

    QML interactions



    So, here our game draws its scene, on top - Qt draws its own controls, but so far all this does not communicate with each other. You can’t live like that.

    If you write your code in QtCreator, or in another IDE, to which, by some miracle, the call to the Qt code generator (MOC) is screwed on, then your life will be simple. It is enough to interconnect slots and signals by name, and QML will receive calls from C ++, and vice versa.

    CodeRush indicates that MOC is easy enough to screw to any of the popular IDEs, since Qt has a means of generating projects from .pro files:

    For VS, a project from a .PRO file is generated like this:

    qmake -tp vc path / to / project / file.pro

    For Xcode , like this:

    qmake -spec macx-xcode path / to / project / file.pro

    al_sh reminds about Add-In for Visual Studio 2013 & 2015

    However, you may want to live without MOC. It is possible! But you have to get some crutches from the zashnik.

    This way (QML -> C ++)



    Qt now supports two ways to link signals and slots - the old one by name and the new one by pointer. So, QML can only be contacted by name. This means, firstly, that you can’t hang a lambda on a signal from QML (whimper-whimper, and I wanted C ++ 11 so much!), And secondly, that you have to have an object in which a slot is declared, and this object must to be the heir to QObject, and inside yourself to have the macro Q_OBJECT, for code generation. And we do not have code generation. What to do? That's right, take objects in which all slots are already declared, and therefore they do not need code generation.

    In fact, this is generally a very useful approach, which, with some probability, you will need. We will use the helper class QSignalMapper. This class has exactly one slot - map (). You can bind to it any number of signals from any number of objects. In response, QSignalMapper for each received signal will generate another signal - mapped (), adding to it a pre-registered ID of the object that generated the signal, or even a pointer to it. How to use it? Very simple.

    We create a separate QSignalMapper for each type of signal that can come from QML (clicked - for buttons, etc.). Further, when in C ++ we need to subscribe to a signal from an object in QML, we associate this signal with the desired QSignalMapper, and we already associate its signal mapped () with our class, or even a lambda (at this level, C ++ 11 already works , cheers cheers). The object ID will come to us, and from it we will understand what we should do with it:

    QObject *b1 = m_qt_wnd->rootObject()->findChild( "b1" );
    QObject::connect( b1, SIGNAL( clicked() ), &m_clickMapper, SLOT( map() ) );
    QObject *b2 = m_qt_wnd->rootObject()->findChild( "b2" );
    QObject::connect( b2, SIGNAL( clicked() ), &m_clickMapper, SLOT( map() ) );
    m_clickMapper.setMapping( b1, "b1" );
    m_clickMapper.setMapping( b2, "b2" );
    QObject::connect( &m_clickMapper, static_cast(&QSignalMapper::mapped), [this]( const QString &sender ) {
        if ( sender == "b1" )
            m_speed *= 2.0f;
        else if ( sender == "b2" )
            m_speed /= 2.0f;
    } );
    


    In the code of the test project, the link to which you will find at the end of the article, there is an example of wrapping this mechanism for a little more convenient use.

    Zifix points out that there is another way of interacting with QML -> C ++ - to throw a C ++ object into QML, and pull it from there. To do this, the object must be a QObject descendant and contain the Q_OBJECT macro processed by the code generator, and also add it to the context:

    SignalsHub signal;
    engine.rootContext()->setContextProperty(QLatin1String("signal"), &signal);
    



    There (C ++ -> QML)



    Here we are faced with an ambush without code generation - it will not work to connect a signal from C ++ to a slot in QML (more precisely, there are methods, but for my taste, they are too complicated). On the other hand, why?

    In fact, we have as many as two (well, OK, one and a half) ways. First, you can directly change the properties of QML objects from C ++ code, I call them setProperty ("propName", value) . That is, if you just need to put down a new text to some field, then you can. Obviously, this method of interaction is quite limited in all senses, but in fact you can’t even imagine how much. The fact is that an attempt to touch the properties of QML objects from a render thread will lead to an error. That is, of these same before / afterRendering you can’t touch anything. Have you already written game logic there? :) I - yes.

    What to do Firstly, you can start a timer in the main thread that will fire once every N seconds and process the game logic. And let the render be rendered separately. We'll have to somehow synchronize them, but this is a solved issue.

    But if you don’t want to do this, then there is a way out! We cannot send QML signals, we cannot write property, but we can even call functions all of a sudden. Therefore, if you need to influence the UI, then it is enough to declare a function in it that your effect will implement (say, setNewText), and then call it from C ++ via invokeMethod:

    QVariant a1 = "NEW TEXT";
    m_fps_label->metaObject()->invokeMethod( m_fps_label, "setText", Q_ARG(QVariant, a1) );
    


    An important point: the arguments in this call can only be of type QVariant, and you need to use this macro, Q_ARG. Also, if the method can return something, then you will need to specify Q_RETURN_ARG (QVariant, referenceToReturnVariable).

    tzlom clarifies that this way you can call not only functions, but also signals and slots declared in QML. At the same time, if you specify the Qt :: QueuedConnection parameter, the call will be delayed, in the stream where you can do it exactly

    Zifix says that the technique described above with throwing a C ++ object in QML can be used to bind signals in the direction C ++ -> QML. This will avoid the search for QML-objects in C ++ code, that is, reduce the bad connection "by name" between C ++ and QML.

    // Так
    Connections {
        target: signal // signal - это имя нашего C++ - объекта, прокинутого в контекст
        onHello: {
            console.log("Hello from С++!");          
        });
    }
    // Или так
    Component.onCompleted: {
        signal.onHello.connect(function() { // связываем сигнал C++-объекта с анонимной ф-ей в QML
            console.log("Hello from С++!");          
        });
    }
    



    Resources



    In principle, everything is already almost fine with us. And if all the games had all the resources lying simply in the open form next to the executable file, the article could be completed on this. Unfortunately, life is a little bit wrong: resources in games are often packed, and even in some special format of their own. Which game engine can efficiently stream from disk to memory and all that.

    There is a desire to push all the resources associated with the UI to the same place where the rest of the game’s resources are. Moreover, they cannot always be clearly separated - sometimes the same texture can be used both in the 3D scene and in the UI. At the same time, I really want that in the QML-file we still wrote “source: images / button_up.png”, so that during development, while our resources are not packed, we could edit the UI in Qt Designer without doing writing plugins for it.

    And at this moment we are waiting for the most severe, and very offensive bummer. In fact, we need to slip Qt our resource system under the guise of a file system. But support for virtual file systems in the form of QAbstractFileEngine in version 5.x was successfully cut out "due to performance problems" (discussion ). I do not know what and with what heel it was written there. All our games work fine with VFS, combining several sources of resources, and do not complain about performance. The most annoying thing is that Qt authors did not propose a replacement.

    However, so far this class has not been completely cut, but only “privatized”, so if you like to live risky, you can use it by connecting a private library and a header.

    The authors left one crutch - in QMLEngine you can register QQuickImageProvider . With it, you can at least load textures from your system.

    In order for QMLEngine to use your QQuickImageProvider and not go directly to the file, you must specify the image path in the QML file not just “images / button_up.png”, but “image: /my_provider/images/button_up.png” (where “my_provider” Is the name with which you registered your QQuickImageProvider successor in QMLEngine). Obviously, if you do this, then you will immediately stop seeing pictures in Qt Designer, which does not know anything about your custom provider and does not want to know.

    There is no crutch that could not be supported by another crutch! You can register another class in QMLEngine - QQmlAbstractUrlInterceptor.Through this Interceptor are all the URLs that are loaded in the process of processing the QML file. And then you can replace them with something. What we need! As soon as we see that the URL type is UrlString, and, for reliability, the URL itself contains the text ".png", then we immediately do:

    QUrl result = path;
    QString short_path = result.path().right( result.path().length() - m_base_url.length() );
    result.setScheme( "image" );
    result.setHost( "my_provider" );
    result.setPath( short_path );
    return result;
    


    setScheme is for QML to understand that you need to look for a suitable ImageProvider
    setHost - the name of our provider
    setPath - but here you need to clarify. The fact is that in Interceptor URLs come already supplemented with the base url of our QMLEngine. By default, this is QDir :: currentPath. Obviously, this is completely inconvenient for us, so we have to cut off an unnecessary piece of the path so that instead of some kind of "file: /// C: /Work/Test/images/button_up.png" we get, as a result, "image: / my_provider /images/button_up.png ".

    Resources 2 - False Trace



    In order to amuse the audience, I’ll tell you how I tried to trick Qt, and load ALL resources from my system.

    QMLEngine also contains a third type of class that you can install for it — NetworkAccessManagerFactory . The indigestible name hides the ability to set your own http request handler. But what if, I thought, we will replace requests for QML files in QQmlAbstractUrlInterceptor with http requests, and in our NetworkAccessManagerFactory (more precisely, in NetworkAccessManager and NetworkReply) actually open files from our resource system?

    The plan worked almost to the very end :) URLs are intercepted, http requests are replaced, even qml files are loaded successfully. But only when trying to read the contents of the qmldir service file with http QQMLTypeLoader does assert :( And I could not get around this behavior. And without this, the whole idea is useless - we will not be able to import our QML modules from our resource system.

    Redux Resources



    By the way, Qt has its own resource system! It allows you to compile resources into an rcc file, and then use them from there. For this, deep in the depths of Qt, its own virtual file system is made, which, if the resource is prefixed with qrc: / or even simply: /, loads it not from the disk, but from where it is needed. Unfortunately, “where it should be” is still not from our resource system.

    There are two ways to register a resource source. Both are calls to different overloads of the QResource :: registerResource static function. The first accepts the name of the resource file on disk. Everything is clear here - we read it from the disk and use it. The second - takes a bare pointer to a certain rccData. The documentation at this point succinctly states that this function registers rccData as a resource. And further still grinds some nonsense about files. This is the result of an unsuccessful copy-paste, wandering from version to version without changes.

    Examination of the source code of the second registerResource overload showed that it does accept the contents of the rcc file as input. Why is data size not passed along with the pointer? It turns out - because Qt does not want to check anything, but wants read-read-read and access violation. At this point, the library expects to receive high-quality binary data that has at least a heading (magic letters “qres” and data about the size and other properties of the remaining part of the memory block). Until the valid header is read, Qt will cheerfully read any memory you slip into it. Not very reliable, but okay.

    It would seem that this option suits us - you can read the rcc-file from our resource system, put it into a QResource, and then use all resources with the qrc: / prefix without any problems. This is partly true. But remember that before registering data in a resource system, you have to completely load it into memory. So stuffing all UI textures into one rcc is probably a bad idea. You will either have to prepare a separate set for each screen, or, for example, put only QML files in rcc, and load pictures from your resource system using the method described above via Interceptor + ImageProvider.

    Release preparation



    If you think that after you have overcome all the Qt software problems, written your code, drew a beautiful UI and packed your resources, everything is ready for release - then this is not entirely true.

    The fact is that Qt is a lot of many DLLs and QML modules. In order to distribute your program, you have to carry all this stuff with you. But in order to carry it, you must first find it, and it is hidden in the corners of the huge Qt installation directory. Qt Creator will find everything and put it where it is necessary, but if we still use a different IDE ... Cutting out all the necessary DLLs and other files with our hands is a tedious and tedious task, and most importantly, it is easy to make a mistake.

    Here, the authors of Qt made a move towards simple programmers, and provided tools such as windeployqt and androiddeployqt. For each platform, such a tool of its own, with its own keys, and behaves differently. For example, windeployqt takes the input path to your main executable file and the directory with your QML files, and at the output it simply copies all the necessary DLL and other to the specified location. Then do it yourself, yourself, yourself.

    But androiddeployqt is the same harvester that deals with the assembly of the APK-package, and hell knows what. On iOS, the situation is similar.

    conclusions



    So, is it possible to use QtQuick / QML to create a UI in games? My short experience of integrating and using this library has shown that in principle it is possible. But much depends on specific goals and limitations.

    Let's say if you are ready to use QtCreator for development - a significant part of the minor inconvenience automatically disappears, but if for some reason you want to stay with your beloved Visual Studio, Xcode or vi, then you need to get ready for some pain.

    If you are developing a game for PC, or it is a very large mobile project with hundreds of megabytes of resources (after all, there are such ones), then 25-40 MB libraries are not a problem for you. If you are writing another casual game for Android, and even with an eye on the Chinese or Iranian markets, with their recommended 50MB per application, then you should think three times before taking up most of this not too useful load.

    However, if you are desperate not to write your own UI library, then QtQuick / QML, as it seems to me, outperforms its competitors in performance, if not in size and not in ease of use.

    Integration of Qt into the project is not too complicated, but it can force to change the logic of the main loop and initialization. In a new project, this can almost certainly be survived, but it is unlikely that changing the UI from another to QtQuick / QML will be successful without much suffering.

    The Qt documentation is pretty good, but in some places it's lying or incomplete. In these cases, you have to go into the source code - and it is very good that it is completely open! Its volumes are solid, but in fact it’s quite possible to figure out how something loads or works.

    Another disadvantage, in comparison with Scaleform and Coherent, is that Scaleform allows you to create interfaces for designers in familiar Adobe programs, and Coherent allows you to hire an HTML specialist to develop a UI. Developing a UI in QML will require the collaboration of a programmer and designer. However, in the end, it all the same comes to this when problems with the performance and behavior of the UI inside the game begin.

    In general, you have to decide, as usual, yourself!

    You can take the code for Qt integration with Nya engine on GitHub MaxSavenkov / nya_qt .

    Also popular now: