Porting Quake3

In the Embox operating system (of which I am the developer) some time ago support for OpenGL appeared, but there was no sensible performance check, only drawing scenes with several graphic primitives.
I have never been particularly interested in game devs, although, of course, I like the games, and decided that this is a good way to have fun, but at the same time check OpenGL and see how the games interact with the OS.
In this article I will talk about how to build and run Quake3 on Embox.
More precisely, we will launch not Quake3 itself , but ioquake3 based on it , which also has open source code. For simplicity, we will call ioquake3 just a quake :)
At once I will make a reservation that the article does not analyze the Quake source code itself and its architecture (you can read about it here , there are translations in Habré ), and in this article we will focus on how to ensure the launch of the game on the new operating system.
The code snippets cited in the article are simplified for a better understanding: missing error checks, using pseudo-code, and so on. The original source can be found in our repository .
Dependencies
Oddly enough, not many libraries are needed to build Quake3. We will need:
- POSIX + LibC -
malloc()
/memcpy()
/printf()
and so on - libcurl - networking
- Mesa3D - OpenGL support
- SDL - support for input devices and audio
With the first paragraph, and so everything is clear - it is difficult to do without these functions when developing in C, and the use of these calls is quite expected. Therefore, support for these interfaces is in one way or another practically in all operating systems, and in this case almost no functionality was added. That had to deal with the rest.
libcurl
It was the easiest. Libc is enough to build libcurl (of course, some features will not be available, but they will not be required). Configuring and building this library is statically very simple.
Usually both applications and libraries are linked dynamically, but since in Embox, the main mode is linking into one image, we will link everything statically.
Depending on the build system used, the specific steps will differ, but the meaning is something like this:
wget https://curl.haxx.se/download/curl-7.61.1.tar.gz
tar -xf curl-7.61.1.tar.gz
cd curl-7.61.1
./configure --enable-static --host=i386-unknown-none -disable-shared
make
ls ./lib/.libs/libcurl.a # Вот с этим и будем линковаться
Mesa / OpenGL
Mesa is an open source framework for working with graphics, a number of interfaces are supported (OpenCL, Vulkan and others), but in this case we are interested in OpenGL. Porting such a large framework is the topic of a separate article. I will confine myself only to the fact that Embox Mesa3D already exists in OS :) Of course, any implementation of OpenGL will work here.
SDL
SDL is a cross-platform framework for working with input devices, audio and graphics.
So far, we are slaughtering everything except graphics, and for frame rendering, we will write stub functions to see when they start to be called.
Backends for working with graphics are set in SDL2-2.0.8/src/video/SDL_video.c
.
It looks like this:
/* Available video drivers */static VideoBootStrap *bootstrap[] = {
#if SDL_VIDEO_DRIVER_COCOA
&COCOA_bootstrap,
#endif#if SDL_VIDEO_DRIVER_X11
&X11_bootstrap,
#endif
...
}
In order not to bother with the "normal" support of the new platform, just add your VideoBootStrap
For simplicity, you can take something as a basis, for example, src/video/qnx/video.c
or src/video/raspberry/SDL_rpivideo.c
, but first we will make the implementation almost empty at all:
/* SDL_sysvideo.h */typedefstructVideoBootStrap
{constchar *name;
constchar *desc;```
int (*available) (void);
SDL_VideoDevice *(*create) (int devindex);
} VideoBootStrap;
/* embox_video.c */static SDL_VideoDevice *createDevice(int devindex){
SDL_VideoDevice *device;
device = (SDL_VideoDevice *)SDL_calloc(1, sizeof(SDL_VideoDevice));
if (device == NULL) {
returnNULL;
}
return device;
}
staticintavailable(){
return1;
}
VideoBootStrap EMBOX_bootstrap = {
"embox", "EMBOX Screen",
available, createDevice
};
Add your own VideoBootStrap
to the array:
/* Available video drivers */static VideoBootStrap *bootstrap[] = {
&EMBOX_bootstrap,
#if SDL_VIDEO_DRIVER_COCOA
&COCOA_bootstrap,
#endif#if SDL_VIDEO_DRIVER_X11
&X11_bootstrap,
#endif
...
}
In principle, at this stage it is already possible to compile SDL. As with libcurl, the compilation details will depend on the particular build system, but somehow you need to do something like this:
./configure --host=i386-unknown-none \--enable-static \--enable-audio=no \--enable-video-directfb=no \--enable-directfb-shared=no \--enable-video-vulkan=no \--enable-video-dummy=no \--with-x=no
make
ls build/.libs/libSDL2.a # Этот файл нам и нужен
Putting yourself Quake
Quake3 assumes the use of dynamic libraries, but we will link it statically, like everything else.
To do this, set some variables in the Makefile.
CROSS_COMPILING=1
USE_OPENAL=0
USE_OPENAL_DLOPEN=0
USE_RENDERER_DLOPEN=0
SHLIBLDFLAGS=-static
First start
For simplicity, we will run on qemu / x86. To do this, you need to install it (hereinafter, there will be commands for Debian, for other distributions packages may be called differently).
sudo apt install qemu-system-i386
And the launch itself:
qemu-system-i386 -kernel build/base/bin/embox -m 1024 -vga std -serial stdio
However, when you start Quake, we immediately get an error
> quake3
EXCEPTION [0x6]: error = 00000000
EAX=00000001 EBX=00d56370 ECX=80200001 EDX=0781abfd
GS=00000010 FS=00000010 ES=00000010 DS=00000010
EDI=007b5740 ESI=007b5740 EBP=338968ec EIP=0081d370
CS=00000008 EFLAGS=00210202 ESP=37895d6d SS=53535353
The error is not displayed by the game, but by the operating system. Debug showed that this error is caused by incomplete SIMD support for x86 in QEMU: some instructions are not supported and generate an exception for an unknown command (Invalid Opcode).upd: As suggested in the WGH comments , the problem was actually that I forgot to explicitly enable SSE support in cr0 / cr4, so QEMU is fine.
This happens not in the Quake, and OpenLibM (a library that we use to implement mathematical functions - sin()
, expf()
and the like). OpenLibm patches so that it __test_sse()
doesn’t do a real check on SSE, but simply think that there is no support.
The above steps are enough to start, in the console you can see the following output:
> quake3
ioq3 1.36 linux-x86_64 Nov 12018
SSE instruction setnot available
----- FS_Startup -----
We are looking in the currentsearchpath:
//.q3a/baseq3
./baseq3
----------------------0 files in pk3 files
"pak0.pk3" is missing. Please copy it from your legitimate Q3 CDROM. PointRelease files are missing. Please re-install the 1.32pointrelease. Alsocheck that your ioq3 executable isin the correct place and that every file in the "baseq3
" directory is present and readable
ERROR: couldn't open crashlog.txt
Already well, Quake3 is trying to start and even displays an error message! As you can see, it lacks the files in the directory baseq3
. It contains sounds, textures and all that. Note, pak0.pk3
must be taken from a licensed CD (yes, open source does not imply free use).
Disc preparation
sudo apt install qemu-utils
# Создаём qcow2-образ
qemu-img create -f qcow2 quake.img 1G
# Добавляем модуль nbd
sudo modprobe nbd max_part=63# Форматируем qcow2-образ и пишем туда нужные файлы
sudo qemu-nbd -c /dev/nbd0 quake.img
sudo mkfs.ext4 /dev/nbd0
sudo mount /dev/nbd0 /mnt
cp -r path/to/q3/baseq3 /mnt
sync
sudo umount /mnt
sudo qemu-nbd -d /dev/nbd0
Now you can transfer the block device to qemu
qemu-system-i386 -kernel build/base/bin/embox -m 1024 -vga std -serial stdio -hda quake.img
When the system starts, we will bounce the disk on /mnt
and run quake3 in this directory, this time it will crash later.
> mount -t ext4 /dev/hda1 /mnt
> cd /mnt
> quake3
ioq3 1.36 linux-x86_64 Nov 12018
SSE instruction set not available
----- FS_Startup -----
We are looking in the current search path:
//.q3a/baseq3
./baseq3
./baseq3/pak8.pk3 (9 files)
./baseq3/pak7.pk3 (4 files)
./baseq3/pak6.pk3 (64 files)
./baseq3/pak5.pk3 (7 files)
./baseq3/pak4.pk3 (272 files)
./baseq3/pak3.pk3 (4 files)
./baseq3/pak2.pk3 (148 files)
./baseq3/pak1.pk3 (26 files)
./baseq3/pak0.pk3 (3539 files)
----------------------
4073 files in pk3 files
execing default.cfg
couldn't exec q3config.cfg
couldn't exec autoexec.cfg
Hunk_Clear: reset the hunk ok
Com_RandomBytes: using weak randomization
----- Client Initialization -----
Couldn't read q3history.
----- Initializing Renderer ----
-------------------------------
QKEY building random string
Com_RandomBytes: using weak randomization
QKEY generated
----- Client Initialization Complete -----
----- R_Init -----
tty]EXCEPTION [0xe]: error = 00000000
EAX=00000000 EBX=00d2a2d4 ECX=00000000 EDX=111011e0
GS=00000010 FS=00000010 ES=00000010 DS=00000010
EDI=0366d158 ESI=111011e0 EBP=37869918 EIP=00000000
CS=00000008 EFLAGS=00010212 ESP=006ef6ca SS=111011e0
EXCEPTION [0xe]: error = 00000000
This error is again with SIMD in Qemu.upd: As suggested in the WGH comments , the problem was actually that I forgot to explicitly enable SSE support in cr0 / cr4, so QEMU is fine. This time, the instructions are used in the Quake3 virtual machine for x86. The problem was solved by replacing the implementation for x86 with an interpreted VM (in more detail about the Quake3 virtual machine and, in principle, about architectural features, you can read everything in the same article ). After that, our functions for the SDL begin to be called, but, of course, nothing happens, because these functions do nothing so far.
Add graphics support
static SDL_VideoDevice *createDevice(int devindex){
...
device->GL_GetProcAddress = glGetProcAddress;
device->GL_CreateContext = glCreateContext;
...
}
/* Здесь инициализируем OpenGL-контекст */SDL_GLContext glCreateContext(_THIS, SDL_Window *window){
OSMesaContext ctx;
/* Здесь делаем ОС-зависимую инициализацию -- мэпируем видеопамять и т.п. */
sdl_init_buffers();
/* Дальше инициализируем контекст Mesa */
ctx = OSMesaCreateContextExt(OSMESA_BGRA, 16, 0, 0, NULL);
OSMesaMakeCurrent(ctx, fb_base, GL_UNSIGNED_BYTE, fb_width, fb_height);
return ctx;
}
The second handler is needed to tell SDL which functions to call when working with OpenGL.
To do this, we start an array and from launch to launch we check which calls are missing, something like this:
staticstruct {char *proc;
void *fn;
} embox_sdl_tbl[] = {
{ "glClear", glClear },
{ "glClearColor", glClearColor },
{ "glColor4f", glColor4f },
{ "glColor4ubv", glColor4ubv },
{ 0 },
};
void *glGetProcAddress(_THIS, constchar *proc){
for (int i = 0; embox_sdl_tbl[i].proc != 0; i++) {
if (!strcmp(embox_sdl_tbl[i].proc, proc)) {
return embox_sdl_tbl[i].fn;
}
}
printf("embox/sdl: Failed to find %s\n", proc);
return0;
}
After a few restarts, the list becomes full enough to draw a splash screen and a menu. Fortunately, Mesa has all the necessary functions. The only thing is that for some reason there is no function glGetString()
, I had to use it instead _mesa_GetString()
.
Now when you start the application, the splash screen appears, hurray!

Adding Input Devices
Add keyboard and mouse support to the SDL.
To work with events you need to add a handler.
static SDL_VideoDevice *createDevice(int devindex){
...
device->PumpEvents = pumpEvents;
...
}
Let's start with the keyboard. We hang up the function to interrupt key press / release. This function should memorize the event (in the simplest case, we simply write to a local variable, if desired, you can use queues), for simplicity, we will only store the last event.
staticstruct input_event last_event;
staticintsdl_indev_eventhnd(struct input_dev *indev) {
/* Пока есть новые события, переписываем ими last_event */while (0 == input_dev_event(indev, &last_event)) { }
}
Then we pumpEvents()
process the event and pass it to the SDL:
staticvoidpumpEvents(_THIS) {
SDL_Scancode scancode;
bool pressed;
scancode = scancode_from_event(&last_event);
pressed = is_press(last_event);
if (pressed) {
SDL_SendKeyboardKey(SDL_PRESSED, scancode);
} else {
SDL_SendKeyboardKey(SDL_RELEASED, scancode);
}
}
SDL uses its own enum for key codes, so you have to convert the OS key code to SDL code.
The list of these codes is defined in the file. SDL_scancode.h
For example, the ASCII code can be converted like this (not all ASCII characters are here, but these are enough):
staticint key_to_sdl[] = {
[' '] = SDL_SCANCODE_SPACE,
['\r'] = SDL_SCANCODE_RETURN,
[27] = SDL_SCANCODE_ESCAPE,
['0'] = SDL_SCANCODE_0,
['1'] = SDL_SCANCODE_1,
...
['8'] = SDL_SCANCODE_8,
['9'] = SDL_SCANCODE_9,
['a'] = SDL_SCANCODE_A,
['b'] = SDL_SCANCODE_B,
['c'] = SDL_SCANCODE_C,
...
['x'] = SDL_SCANCODE_X,
['y'] = SDL_SCANCODE_Y,
['z'] = SDL_SCANCODE_Z,
};
Everything is on the keyboard, the SDL and Quake itself will do the rest. By the way, about here it turned out that somewhere in the processing of quake keystrokes uses instructions that are not supported by QEMU, you have to switch to the interpretable virtual machine from the virtual machine for x86, to do this, add it BASE_CFLAGS += -DNO_VM_COMPILED
to the Makefile.
After that, finally, you can solemnly “skip” the screensavers and even start the game (with some error messages :)). It was pleasantly surprised that everything is drawn as it should, albeit with very low fps.
Now you can start to support the mouse. To interrupt the mouse, another handler will be needed, and event handling will require some complication. We confine ourselves only to the left mouse button. It is clear that in the same way you can add the right key, wheel, etc.
staticvoidpumpEvents(_THIS) {
if (from_keyboard(&last_event)) {
/* Здесь наш старый обработчик клавиатуры */
...
} else {
/* Здесь будем обрабатывать события мыши */if (is_left_click(&last_event)) {
/* Зажата левая клавиша мыши */
SDL_SendMouseButton(0, 0, SDL_PRESSED, SDL_BUTTON_LEFT);
} elseif (is_left_release(&last_event)) {
/* Отпущена левая клавиша мыши */
SDL_SendMouseButton(0, 0, SDL_RELEASED, SDL_BUTTON_LEFT);
} else {
/* Перемещение мыши */
SDL_SendMouseMotion(0, 0, 1,
mouse_diff_x(), /* Сюда передаём горизонтальное смещение мыши */
mouse_diff_y()); /* Сюда передаём вертикальное смещение мыши */
}
}
}
After that, it is possible to control the camera and shoot, hooray! In fact, this is enough to play :)
Optimization
Cool, of course, that there is a control and some kind of graphics, but such an FPS is completely worthless. Most likely, most of the time is spent on OpenGL (and it is software, and, moreover, SIMD is not used), and the implementation of hardware support is too long and difficult task.
We will try to speed up the game with a little blood.
Compiler optimization and resolution reduction
We collect the game, all the libraries and the OS itself with -O3
(if, all of a sudden, someone has scanned this place, but does not know what kind of flag it is - read more about the GCC optimization flags here ).
In addition, we use the minimum resolution - 320x240, to facilitate the work of the processor.
KVM
KVM (Kernel-based Virtual Machine) allows you to use hardware virtualization (Intel VT and AMD-V) to improve performance. Qemu supports this mechanism, you need to do the following to use it.
First, you need to enable virtualization support in the BIOS. My motherboard is Gigabyte B450M DS3H, and AMD-V is enabled via MIT -> Advanced Frequency Settings -> Advanced CPU Core Settings -> SVM Mode -> Enabled (Gigabyte, what's wrong with you?).
Then we put the necessary package and add the appropriate module.
sudo apt install qemu-kvm
sudo modprobe kvm-amd # Или kvm-intel
Everything, now it is possible to transfer qemu a flag -enable-kvm
(or -no-kvm
, not to use hardware acceleration).
Total
The game has started, the graphics are displayed as needed, the control is working. Unfortunately, the graphics are drawn on the CPU in one stream, also without SIMD, because of the low fps (2-3 frames per second) it is very inconvenient to manage.
The porting process was interesting. Maybe in the future it will be possible to run quake on a platform with hardware graphic acceleration, but for now I’ll stop on what is.