DOOM 3 BFG Source Code Review: Introduction (Part 1 of 4)

Part 1: Introduction.
Part 2: Multithreading
Part 3: Rendering (Note translation - during translation)
Part 4: Doom classic - integration (Note translation - during translation)

November 26, 2012 ID Software released Doom 3 BFG edition source code (in total month after the appearance of the game on store shelves). The idTech4 engine, which is almost 10 years old, has been updated with the solutions used in idTech 5 ( Rage is the first game on this engine), and it was very interesting to get acquainted with its source code.

I would call the idTech4 enhanced engine, because this is essentially idTech4, but using idTech5 elements:

• Threading system
• Sound system
• Resource Management System

By far the most attractive aspect is the flow control system: Doom 3 was developed at the dawn of the era of multi-core systems, when few computers still used SMP . Now the rules have changed, even phones are equipped with several cores and the game engine must be multi-threaded to use the full potential of the machine.

I hope this inspires people to understand the source code, improve their skills and become better engineers.

First contact

Acquaintance with Doom 3 BFG is impressive because to start from source you need to do only 2 steps:

1. Get the sources located on GitHub:
   

    git clone https://github.com/id-Software/DOOM-3-BFG
   

   
   
2. Open Visual Studio 2010 Express and press F8 to compile. Done!

Note: If the Direct3D SDK is installed, the complete project will compile in less than a minute with 5 minimal warnings.

Debug mode

Only 3 steps are needed to start tinkering in Visual Studio 2010 Express:

1. In the debug command line, specify the base path:
   

    +set fs_basepath "C:\Program Files\Steam\SteamApps\common\DOOM 3 BFG Edition" +set r_fullscreen 0
   

   
   
2. Open the Doom3BFG project.
3. Press F5

Readability of source code

A subset of C ++:

Doom 3 BFG is written in C ++, a language so great that it can be used both to create great code, and for such an abomination that your eyes will bleed . Fortunately, ID Software used a subset of the C ++ language close to “C with classes”, which would be less difficult to grasp:

• There are no exceptions.
• No links (pointers used)
• Minimal use of templates
• Constants everywhere
• Classes
• Polymorphism
• Inheritance

And despite multithreading, the code does not use smart pointers or Boost . What a relief (after all, this is what usually makes the code unreadable).

Comments

There are many comments and they are quite useful, as they, as a rule, describe with one sentence what goes through in the most important places of the next block. Here is an example from ParallelJobList.cpp:

    int idJobThread::Run() {
	threadJobListState_t threadJobListState[MAX_JOBLISTS];
	int numJobLists = 0;
	int lastStalledJobList = -1;
	while ( !IsTerminating() ) {
    // fetch any new job lists and add them to the local list
		if ( numJobLists < MAX_JOBLISTS && firstJobList < lastJobList ) {
			threadJobListState[numJobLists].jobList = jobLists[firstJobList & ( MAX_JOBLISTS - 1 )].jobList;
			threadJobListState[numJobLists].version = jobLists[firstJobList & ( MAX_JOBLISTS - 1 )].version;
			threadJobListState[numJobLists].signalIndex = 0;
			threadJobListState[numJobLists].lastJobIndex = 0;
			threadJobListState[numJobLists].nextJobIndex = -1;
			numJobLists++;
			firstJobList++;
		}
    // if the priority is high then try to run through the whole list to reduce the overhead
    // otherwise run a single job and re-evaluate priorities for the next job
    bool singleJob = ( priority == JOBLIST_PRIORITY_HIGH ) ? false : jobs_prioritize.GetBool();
    // try running one or more jobs from the current job list
    int result = threadJobListState[currentJobList].jobList->RunJobs( threadNum, threadJobListState[currentJobList], singleJob );

In general, the reader gets a direct understanding of each part of the algorithm, and I hope that this will inspire people to write better code: after all, now developing software, it is not important to be a big ace than others. It is equally important to be able to work in a team, creating code:

• Elegantly engineered
• Easy to read, using comments where appropriate

Doom 3 BFG has high rates in both of these positions.

What changed?

• 2 “Game” projects (Doom III classic and Ressurection) combined into one project
• Removed cUrl
• Removed the stupid name DoomDLL ... It was actually generating DOOM3.EXE
• Removed obsolete Maya tools for exporting md5 models, animations and camera paths.
• Removed TypeInfo, instead of it added RTTI / Introspection.

The solution explorer in Visual Studio has become noticeably cleaner (before and after):

Doom 3 BFG Subprojects

Projects	Builds	Observations
Amplitude	Amplitude.lib	Used in Doom Classic: A tool for adjusting the amplitude of the WAV.
Doom3bfg	Doom3BFG.exe	Engine Doom 3 BFG.
doomclassic	doomclassic.lib	Seriously redesigned Doom1 / 2 engine.
external	external.lib	Sources of jpeg-6 and zlib.
Game-d3xp	Game-d3xp.lib	A single library of the game, including the original game + extensions + new levels. Note that now it is going to a static library instead of a DLL.
idLib	idLib.lib	The id software toolkit for working with the file system.
timidity	timidity.lib	Used in Doom Classic to convert MIDI files to WAV format.

New architecture

The architecture is significantly different from the original Doom III: now everything is compiled into one monolithic executable file (the original Doom III was compiled into one executable file and one DLL containing game logic). This was done for two reasons (according to the main developer Brian Harris):

• Consoles such as the PS3 / Xbox360 do not support dynamic libraries in the best way.
• Speed ​​up development. When using the dll library there are problems with memory allocation. This causes errors that are difficult to track.

Changes related to development for consoles:

Focusing on the Xbox 360 and PS3 in a project initially focused on PCs has led to many important updates:

1. As previously mentioned, the entire game is contained in one executable file.
2. The game uses PAK files (which are ZIP archives) to store various parts. The high latency of DVD drives pushed ID Software to the following distribution of resources: one file, contains everything you need for one level load.
3. Game assets were textual, but in order to reduce load time, some of the assets, such as models and animations, are now binary (.bmd5mesh and .bmd5anim).
4. Doom 3 was designed to work with a resolution of 640x480 with an aspect ratio of 4: 3. Currently, TVs and monitors most often have an aspect ratio of 16: 9, so all the menus have been re-created. Probably, in order to accelerate development, it is implemented on Adobe Flash. Doom 3 BFG uses its own Flash interpreter (/ neo / swf /). Again, Flash is used to speed development.
5. Shader rendering has been rewritten using GLSL 1.5. HLSL shaders can be converted on the fly.
6. To obtain an acceptable frame rate, the flashlight could not be used with weapons. (Note.– in the original there is the phrase “Now with Doom 3 BFG it can be duck taped on weapons but for performances reason it won't cast any shadows.” Most likely it is not “duck taped”, but “duct taped” "- scotch tape, that is," Now, in Doom 3 BFG, it can be attached to weapons ... ")
7. Because the menu is now cross-platform. The PC has largely lost its rendering settings: we get a simple version that is used both in PCs and consoles.

Multithreading

Over the 10 years between the development of the old and the new engine, a paradigm shift has occurred: the “free cheese” has ended, and game engines must be developed using multi-threading. Therefore, the most attractive thing to read in Doom III BFG is the idTech5 Threading architecture. (A detailed review in the second part of the translation).

Rendering

There are 2 main changes:

• Powered by Open GL 3.2 Profile Compatibility Using GLSL Shaders 1.5.
• The use of multithreading (up to four threads working simultaneously).

(Detailed review in the 3rd part of the translation)

Doom classic

Doom III BFG allows you to play Doom 1 and Doom 2. At first glance, the simple task is to integrate the old Doom1 engine into the new Doom 3 BFG: just redirect all the inputs / outputs! But given the split-screen mode on the PS3 and Xbox360, this is not easy to implement. (A detailed review in the 4th part of the translation)



Fasting is a translation, but since published from the sandbox - I can not change the type of post. Subsequent parts will be framed correctly. Translation errors, typos with pleasure I will correct, write to hp.

The original of the first part - Fabien Sanglard