UI performance comparison in WPF, Qt, WinForms and FLTK

By a measure of UI performance, we mean the number of responses to user actions per unit time. And under the response is the response of the application requested by the user.

The short response time can explain a number of user preferences:

1. Preference for analog interfaces over digital (when there is a delay in processing digital input);
2. At the dawn of Windows, - the preferences of users to work with DOS programs in "text mode", and not with GUI counterparts in Windows (response time in text mode was then noticeably shorter on a similar platform);
3. Preference of real game consoles to their emulators (emulators often have a response time different from the response time of the original consoles);
4. The preference of users of iOS and Android over WinCE and Symbian (among other things, for example, in iOS the goal was quick response and support of 60 FPS, Android, although it did not set such goals, was noticeably more responsive than WinCE and Symbian);
5. In cars, the ambiguous attitude of users towards automatic transmissions, the electronic gas pedal, and some other systems introducing a delay between the control action and reaction to it (this refers to the least advanced versions of these solutions).

A long response time is essentially a “feedback with delay”, which can be read in more detail here: “Feedback with delay in a tap with hot water, a rover and a demographic pyramid . ”

Performance comparison

The delay in response to user input in digital systems is natural and inevitable. However, in the UI for perception, only those delays that the user is able to perceive are of importance. For example, in the perception of visual information, the average user is not able to distinguish delays of less than 1/60 of a second, so a further decrease in the time of the visual response is unlikely to be justified.

Consider the user interface responsiveness of the WPF , Qt , WinForms, and FLTK libraries. It is time-consuming to measure the responsiveness of all controls of these libraries, and it is difficult in each case to measure the interval between the user's input and the reaction of the control to this input. Therefore, we simplify the estimation problem a little. We will test one, but complex, and generally indicative control, which is present in all libraries - DataGrid . We will measure responsiveness by FPS in response to scrolling the contents of this control. In order to avoid counting incomplete frames, we will use double buffering.

While scrolling the grid, one way or another we will check the basic mechanisms of the library that are responsible for the reaction to input, rendering of controls, text, processing of the visual control tree, and partly composition. This will give us the opportunity to assess the potential of the library to work with a loaded UI.

For each library, I prepared a test application that populates the grid with the same data. This data is 5,000 rows of 20 columns with some arbitrary information (I think this number of rows and columns is close to the real maximum needs for displaying complex objects). I did not optimize grid filling, so you should not give importance to the fact that for some libraries it fills slowly. We will measure performance after the grid is full.

I tested performance only under Windows (although libraries, with the exception of WPF, are cross-platform), an FPS counter was written for verification (only for Windows and 32 bit color depth) that determines frame changes at the top of the main screen. The error of the counter can be about 1 frame.

Methodology for measuring performance:
1. Run the counter FPSCounter.exe .
2. Run one of the test applications FltkGrid.exe, FormsGrid.exe, QtGrid.exe or WpfDatagridTest.exe and expand it to the entire main screen (this is necessary because only the change in the top of the frame is detected on the main screen)
3.We move the slider of the vertical scroller grid up and down until it stops, while the slider is moving, we look at the FPS value in the upper left corner of the screen or in the counter window. (to get the maximum FPS values, you need to move the slider quickly, otherwise we will run into our own “performance” rather than UI performance)

I performed FPS measurements on several platforms that were at hand, almost all platforms on Windows 7 x64, other applications for a while measurements closed.

Archive with FPSCounter.exe, FltkGrid.exe, FormsGrid.exe, QtGrid.exe and WpfDatagridTest.exe, as well as Qt libraries required for the Qt test application (16 Mb)

Archive with FPSCounter.exe, FltkGrid.exe, FormsGrid. exe (i.e. without Qt, but very small)

Applications compiled for the x64 platform. In addition, msvs runtime 2010 may be required to run.

results

Below is a table with a list of platforms on which measurements were launched and their results. The table also shows the evaluation of the single thread performance of the platform (Single Thread Performance) in accordance with www.cpubenchmark.net .



In addition to the table, I want to add that Office applications and, for example, the Chrome browser (on “good” pages) show FPS approximately equal to FLTK or slightly less.

For a more visual illustration, I will add a graph of the dependence of FPS on the evaluation of Single Thread performance. The graph is based on the data given in the table.



The graph is worth a little comment. On a platform with a rating of 1000, a low vertical screen resolution was used, which greatly reduced the number of visible cells, thereby significantly increasing the FPS of vertical scrolling. I also want to add that a number of optimizations were disabled to compile FLTK examples, so it’s quite possible to enable them and you can slightly increase the evaluation of the FLTK example on all platforms (for the same reason, the console window appears during the start of the FLTK example).

In general, the dependence of FPS on single-threaded processor performance is fairly linear. There is an assumption that multi-threaded performance is still not very applicable to UI libraries (for example, the superiority of i7 in multi-threaded performance has little effect on FPS). The measurements also showed a weak dependence of FPS in our test on the video card (there is certainly a dependence, but it seems that the video card is not a bottleneck in these tests) Another interesting detail was the restriction of 30 FPS on a number of platforms. I’m not sure if this is due to the video card driver or some of its settings, but in some cases it was not possible to get more than 30 FPS ...

Based solely on the measurement data, I can not recommend FLTK as a universal means of increasing the responsiveness of your UI, especially since writing a UI in FLTK is very time-consuming (it took me about 20 minutes to write the FLTK example, although the rest of the examples were written much faster). However, the measurement results make us think that many libraries do not fully reveal the potential of the iron on which they are executed, making applications less attractive to the user (at least in terms of responsiveness).

I would be glad if someone similarly tests other UI libraries in this way, suggests the option of optimizing the performance of test cases (for the scrolling case under consideration), or just talks about FPS in test cases using my own configuration.

I enclose the source of the counter and tests .

How to get high FPS?

As you can see from the test, stable 60 FPS, in the case of a heavy UI, we get only on expensive hardware and the most expensive UI library to develop (that is, it will be expensive to develop both hardware and the consumption of this hardware), sometimes it’s sometimes worth it, but for now it is rather an exception. However, if you do not go to extremes and set yourself the goal of getting at least 20 FPS in information-loaded interfaces. What will it cost us?

For the reviewed libraries, there are probably not many options:
1. FLTK + is almost the cheapest hardware. We will spend significantly more time on developing the UI, but at current prices we can save ~ $ 100-200 on hardware at the user's workstation.
2.Qt + medium iron. On hardware, you won’t save much, but developing a UI will be cheaper than in the case of FLTK. Probably in some cases the option will be optimal.
3. WPF + expensive iron ie additional 200-300 $ per workplace. After all, if on the i7-3770 we get only 12 frames, then you need at least iron and a half times more powerful. Probably the i7-5930K or perhaps the i7-4790K paired with a good graphics card can handle the 20 FPS task. However, this is unlikely to be an effective solution, and will it cope ... unfortunately there is no such iron at hand to check, but if you extrapolate single-threaded performance, then its estimate should be over 3000, to get 20 FPS at 1280x1024 ... such iron simply does not exist, according to at least here ...
4.Lighten the UI until we hit 20 FPS. For example, if using WPF, instead of 20 columns, only 6 are left in the grid, then on i7-3770 we get stable 20 FPS, but if you leave only 3-4 columns, we get 20 FPS on budget hardware. Reducing the size of the grid itself should also give a positive effect (although it is different in different libraries, and oddly enough, for WPF, the effect is least pronounced). Are such decisions acceptable? Not applicable everywhere, but still covers a number of tasks that do not focus on the presentation of data.

PS: The idea to compare grid performance came up after I came across a low performance WPF grid. In the comments on my previous article, “Choosing Between C ++ and C #,” I specifically addressed this issue.
Then it became interesting to me how alternative libraries cope with the task of displaying the grid, so there appeared test applications and the results described in this article.