Porting Qt to STM32
Good afternoon! We in the Embox project launched Qt on STM32F7-Discovery and would like to tell about it. Earlier, we already told how we managed to launch OpenCV . Qt is a cross-platform framework that includes not only graphical components, but also things like QtNetwork, a set of classes for working with databases, Qt for Automation (including the implementation of IoT), and much more. The developers of the Qt team have foreseen the use of Qt in embedded systems, so the libraries are pretty well configured. However, until recently, few people thought about porting Qt to microcontrollers, probably because this task looks complicated - Qt is large, MCUs are small.
On the other hand, at the moment there are microcontrollers designed to work with multimedia and are superior to the first Pentiums. About a year ago she appeared in Qt blog post . The developers made the Qt port under RTEMS, and launched examples with widgets on several boards running stm32f7. It interested us. It was noticeable, and the developers themselves write about this that Qt slows down on the STM32F7-Discovery. We wondered if we could run Qt under Embox, and not just draw a widget, but start the animation.
Embt has been porting Qt 4.8 for a long time, so we decided to try it. We chose the moveblocks application - an example of springy animation.
Qt moveblocks on QEMU
To begin with, we configure Qt, if possible, with the minimum set of components required to support animation. For this there is the option “-qconfig minimal, small, medium ...”. It includes a configuration file from Qt with many macros - what to enable / disable. After this option, add other flags to the configuration if we want to disable something else. Here is an example of our configuration .
In order for Qt to work, you need to add an OS compatibility layer. One way is to implement QPA (Qt Platform Abstraction). The basis was the ready-made plugin fb_base as part of Qt, on the basis of which QPA for Linux works. The result is a small emboxfb plugin that provides the Embox Qt framebuffer, and then it draws there already without any help.
This is how plugin creation looks like
QEmboxFbIntegration::QEmboxFbIntegration()
: fontDb(new QGenericUnixFontDatabase())
{
struct fb_var_screeninfo vinfo;
struct fb_fix_screeninfo finfo;
const char *fbPath = "/dev/fb0";
fbFd = open(fbPath, O_RDWR);
if (fbPath < 0) {
qFatal("QEmboxFbIntegration: Error open framebuffer %s", fbPath);
}
if (ioctl(fbFd, FBIOGET_FSCREENINFO, &finfo) == -1) {
qFatal("QEmboxFbIntegration: Error ioctl framebuffer %s", fbPath);
}
if (ioctl(fbFd, FBIOGET_VSCREENINFO, &vinfo) == -1) {
qFatal("QEmboxFbIntegration: Error ioctl framebuffer %s", fbPath);
}
fbWidth = vinfo.xres;
fbHeight = vinfo.yres;
fbBytesPerLine = finfo.line_length;
fbSize = fbBytesPerLine * fbHeight;
fbFormat = vinfo.fmt;
fbData = (uint8_t *)mmap(0, fbSize, PROT_READ | PROT_WRITE,
MAP_SHARED, fbFd, 0);
if (fbData == MAP_FAILED) {
qFatal("QEmboxFbIntegration: Error mmap framebuffer %s", fbPath);
}
if (!fbData || !fbSize) {
qFatal("QEmboxFbIntegration: Wrong framebuffer: base = %p,"
"size=%d", fbData, fbSize);
}
mPrimaryScreen = new QEmboxFbScreen(fbData, fbWidth,
fbHeight, fbBytesPerLine,
emboxFbFormatToQImageFormat(fbFormat));
mPrimaryScreen->setPhysicalSize(QSize(fbWidth, fbHeight));
mScreens.append(mPrimaryScreen);
this->printFbInfo();
}
And so here the redrawing will look
QRegion QEmboxFbScreen::doRedraw()
{
QVector rects;
QRegion touched = QFbScreen::doRedraw();
DPRINTF("QEmboxFbScreen::doRedraw\n");
if (!compositePainter) {
compositePainter = new QPainter(mFbScreenImage);
}
rects = touched.rects();
for (int i = 0; i < rects.size(); i++) {
compositePainter->drawImage(rects[i], *mScreenImage, rects[i]);
}
return touched;
}
As a result, with the included optimization of the compiler for memory size -Os, the image of the library turned out to be 3.5 MB, which of course does not fit into the main memory of the STM32F746. As we already wrote in our other article about OpenCV, this board has:
- 1 MB ROM
- 320 Kb RAM
- 8 MB SDRAM
- 16 MB QSPI
Since support for executing code from QSPI has already been added for OpenCV, we decided to start by loading the entire Embox c Qt image into QSPI. And hurray, everything almost immediately started from QSPI! But as with OpenCV, it turned out to be too slow.
Therefore, we decided to do this - first copy the image to QSPI, then load it into SDRAM and execute from there. From SDRAM it got a little faster, but still far from QEMU.
Next was the idea to include a floating point - because Qt does some calculations of the coordinates of the squares in the animation. They tried, but here they did not get visible acceleration, although in the articleQt developers have argued that FPU provides a significant increase in speed for “dragging animation” on the touchscreen. Perhaps moveblocks has significantly fewer floating point calculations, and this depends on a specific example.
The most effective idea was to transfer the framebuffer from SDRAM to internal memory. To do this, we did not have a screen size of 480x272, but 272x272. We also reduced the color depth from A8R8G8B8 to R5G6B5, thus reducing the size of one pixel from 4 to 2 bytes. We got the framebuffer size 272 * 272 * 2 = 147968 bytes. This gave significant acceleration, perhaps the most noticeable, the animation became almost smooth.
The last optimization was the execution of Embox code from RAM, and Qt from SDRAM. To do this, we first, as usual, link statically Embox along with Qt, but we place the segments of text, rodata, data and bss libraries in QSPI, so that we can later copy them to SDRAM.
section (qt_text, SDRAM, QSPI)
phdr (qt_text, PT_LOAD, FLAGS(5))
section (qt_rodata, SDRAM, QSPI)
phdr (qt_rodata, PT_LOAD, FLAGS(5))
section (qt_data, SDRAM, QSPI)
phdr (qt_data, PT_LOAD, FLAGS(6))
section (qt_bss, SDRAM, QSPI)
phdr (qt_bss, PT_LOAD, FLAGS(6))
Due to the execution of the code, Embox from ROM also received tangible acceleration. As a result, the animation turned out to be quite smooth:
Already at the very end, while preparing an article and trying various configurations of Embox, it turned out that Qt moveblocks works fine from QSPI with a framebuffer in SDRAM, and the size of the framebuffer was the bottleneck! Apparently, to overcome the initial “slideshow”, acceleration by 2 times was enough due to the banal reduction of the framebuffer size. But it was not possible to achieve such a result by transferring only Embox code to various fast memory (acceleration was obtained not by 2, but by about 1.5 times).
How to try it yourself
If you have STM32F7-Discovery, you can run Qt under Embox yourself. You can read how to do this on our wiki .
Conclusion
As a result, we managed to launch Qt! The complexity of the task, in our opinion, is somewhat exaggerated. Naturally, you need to take into account the specifics of microcontrollers and generally understand the architecture of computing systems. The optimization results indicate a well-known fact that the bottleneck in the computing system is not the processor, but the memory.
This year we will be participating in the TechTrain festival . There we will tell in more detail and show Qt, OpenCV on microcontrollers and our other achievements.