October 25, 2017 at 09:53Reverse Engineering Strike Commander
- Transfer
In the early 1990s, one company was at the forefront of PC gaming progress: Origin Systems . Her slogan was “We create worlds” and, damn it, they corresponded to it: the Ultima , Crusader and Wing Commander series shocked the imagination of the players.
It took four years and more than a million man-hours to create one of the games: Strike Commander . The famous flight simulator had its own 3D engine called RealSpace , in which technologies first appeared, today taken for granted: texture mapping, Gouro shading, changing the level of detail and dithering colors.
My old dream was to play it in a helmet of virtual reality. Thanks to Oculus Riftthis fantasy has become one step closer to reality.
But, as it turned out, the source code of the game was lost and will never be released, so I decided to do the reverse development.
At the time of publication of this article, my project was not yet completed, but I want to share the techniques that allowed me to make from this:
this:
... and perhaps inspire someone to join my adventure.
In the early 90s, flight simulators were good, but Falcon 3.0 or Flight simulator 4.0 focused more on the accuracy of the flight model, rather than beautiful visualization:
There were many flight simulators on the market and few noticed that in 1990 Origin Systems announced the creation of a new game. But that all changed after CES 1991 in Chicago, where a demo was shown. No one could believe that in the game there would be elements that were present at that time only in flight simulators of the Air Force: texture mapping, Gouraud shading, fog and many others. The screenshots show technologies that are much ahead of their competitors:
After 1991, many began to wait for the game. Not only because of the stunning engine and exciting plot - RealSpace initially supported additional equipment, very expensive and fragile accessories that would be desirable for any fan of flight simulators - THRUSTMASTER WEAPON CONTROL SYSTEM and THRUSTMASTER FLIGHT CONTROL SYSTEM:
You could even plug in the THRUSTMASTER RUDDER:
But that was not all: the game supported a virtual cockpit. The four-way ThrustMaster mini-joystick allows you to move the pilot’s head and monitor ground targets / enemy planes without the need for a virtual reality helmet:
The game and its three-dimensional engine simply “blew up the brain”, but they needed an incredibly powerful machine:
If we draw an analogy with the modern technological level, then the recommended configuration would be this:
And just to buy a game was not enough, you still had to go through the installation process! Having opened the box, the user saw 8 floppy disks (+3 for Speech Pack):
Interesting fact: note that the poster shows the release date: Christmas 1991. The game was only completed in 1993 after a long process that Chris Robert called "The Apocalypse of Today's Computer Games."
Unpacking the game from floppies to the hard drive and changing 13 disks took a good half an hour. And when you already thought that it was over, the game began to generate all the cards. The size of the game increased from 24 MB to 38 MB: four times more than any other game of that time.
An interesting fact: the card was generated from one seed (integer) inserted into the pseudo-random number generation algorithm. It was a skillful technique that avoided increasing the amount of data on disks by generating a card after installation. If you're interested in the details of this trick, I recommend reading The Backroom Boys and the chapter on Frontier Elite.
On the 386th PC, card generation took ONE HOUR. But Origin Systems came up with the idea of supplying with the game box an excellent one-page magazine in the gloomy surroundings of the 2012 game world. Thanks to Sudden Death, the whole process became less painful (note that on page 38 there is a fake ad promising to release Strike Commander by Christmas 2013).
Finally, after these steps, players could enjoy the game ... in April 1993! Despite the fact that the game was delayed for two years, it had everything that Origin promised, and it sold very well. Many spent long nights fighting in battles, even though the game should have looked like this:
... at the minimum settings it looked like this:
In general, this was enough to attract pilots and create good memories.
After the release of Oculus Rift, my interest in Strike Commander increased again: a suitable hardware appeared. Since the game was 20 years old, I expected that the source code was already released, but quickly came across a story about the sad history of Origin Systems withering.
In September 1992, Origin Systems was bought by Electronic Arts and in about 1999 all projects were canceled due to poor sales of Ultima 9. The company had to focus on another area in which it was also the pioneer: MMORPG Ultima Online . Many people believe that the source codes and versions on the "gold" of all finished games are stored somewhere in the depths of the EA repository. But after contacting people from Wing Commander CIC , I found out that all the source code disappeared after the company was closed.
Today it’s hard to believe, but at that time developers and companies were more interested in new games, rather than preserving the “old stuff”, there wasn’t nostalgia factor yet, there wasn’t a big base of fans and shops like modern Good Old Games . Many stories can illustrate the level of “primitiveness” of source control, but perhaps the best story is “ZAP SC” at 15 minutes 14 seconds:
On the first day of work, one developer managed to delete the entire 900 MB Strike Commander source tree. The IT department spent 72 hours restoring everything from the machines of the developers. The interview also mentions that the Wing Commander 1 and 2 code was transferred on floppy disks: before Strike Commander they had no network!
Interesting fact: an unexpected turn of events - part of the source code was recently found by former Origin developers: Wing Commander CIC stores an offline archive that stores the source codes of Wing Commander I and Wing Commander III . People who worked on Ultima 8, the anonymous source was proposed (but rejected) the source code «Ultima 8: Pagan». As for the Strike Commander, I never heard anyone stay with him.
Over the years, many teams managed to work on Origin Systems games and achieved excellent results:
How long and hard can it be? After talking with Gregory Montoir, who did almost the entire reverse development of Another World , I realized that working an hour in the evening can convert 10 KB of assembler code back to C in a month. Strike Commander has a lot of executable files, and the initial discouraging estimate of the time was this: It was frustrating: if someone started work in 1993, they would have finished six years ago, and it would be enough for me to do it ! But by running the IDA and examining the files a bit, I realized that not everything needs to be reversed: the 3D engine is completely in STRIKE.EXE and a small team will be able to handle it in a reasonable amount of time.
The roadmap I originally drew for Strike Commander's reverse engineering was this:
And I got to work.
Strike Commander does not consist of a single monolithic executable. Six executable files work together to provide gameplay. The basic idea is similar to the Second Reality code review . The use of multiple .exe files can be explained in different ways:
After studying in the IDA, it turned out that each executable file uses the DOS 21h system call to load and run others
For research and study of the components responsible for each part of Strike Commander, DosBOX proved to be very useful. The name of the currently running executable is shown in the window title:
Note: if you delete the PAK card archive, it
Important files used by the game:
The PAK and TRE formats are 100% clear thanks to Mario's specifications .
Archives are easy to manage / unpack with TreArchive and PakArchvive in libRealSpace.
Aircraft are stored in complete files in IFF archives. They are 95% understandable. Consist of four key parts:
See JET format specifications .
Maps are 95% understandable. They are stored in
See map texture format specifications .
Menus are 50% understandable (visualization, but not logic).
No specifications yet.
Animations are 50% clear:
See animation format specifications .
Palettes are 100% clear: see palette format specifications .
So far they have not worked on them. They are probably all in exe.
Given the time and documentation found in the playtester’s manual, I think that there’s not a simple decision tree. Probably hierarchical finite state machines.
See a detailed discussion of all the techniques in Arctificial Intelligence for Games .
The format of saved games is 75% understandable: see the specifications for the format of saved games .
The Strike Commander rewriting project has begun. Full source code can be found on GitHub :
Nothing yet! I strongly hoped that the game uses a SCUMM -style virtual machine . This part is performed
The effects of the palette will probably be difficult to recreate: when the pilot experiences too much overload, the screen turns gray (and then dark if it gets even larger G). This is implemented by interpolating between palettes. A shader can help here.
Work on the integration of Oculus Rift has not yet begun. [approx. lane: the last commit on Github of the project is dated October last year, so it seems, alas ...]
It took four years and more than a million man-hours to create one of the games: Strike Commander . The famous flight simulator had its own 3D engine called RealSpace , in which technologies first appeared, today taken for granted: texture mapping, Gouro shading, changing the level of detail and dithering colors.
My old dream was to play it in a helmet of virtual reality. Thanks to Oculus Riftthis fantasy has become one step closer to reality.
But, as it turned out, the source code of the game was lost and will never be released, so I decided to do the reverse development.
At the time of publication of this article, my project was not yet completed, but I want to share the techniques that allowed me to make from this:
this:
... and perhaps inspire someone to join my adventure.
Before Strike Commander
In the early 90s, flight simulators were good, but Falcon 3.0 or Flight simulator 4.0 focused more on the accuracy of the flight model, rather than beautiful visualization:
There were many flight simulators on the market and few noticed that in 1990 Origin Systems announced the creation of a new game. But that all changed after CES 1991 in Chicago, where a demo was shown. No one could believe that in the game there would be elements that were present at that time only in flight simulators of the Air Force: texture mapping, Gouraud shading, fog and many others. The screenshots show technologies that are much ahead of their competitors:
After 1991, many began to wait for the game. Not only because of the stunning engine and exciting plot - RealSpace initially supported additional equipment, very expensive and fragile accessories that would be desirable for any fan of flight simulators - THRUSTMASTER WEAPON CONTROL SYSTEM and THRUSTMASTER FLIGHT CONTROL SYSTEM:
You could even plug in the THRUSTMASTER RUDDER:
But that was not all: the game supported a virtual cockpit. The four-way ThrustMaster mini-joystick allows you to move the pilot’s head and monitor ground targets / enemy planes without the need for a virtual reality helmet:
1993: first contact
The game and its three-dimensional engine simply “blew up the brain”, but they needed an incredibly powerful machine:
- IBM PC 486-DX2 66 MHz
- 4 MB RAM
- At least 38 MB of hard disk space
- The game was delivered on eleven floppy disks of 1.44 MB each.
If we draw an analogy with the modern technological level, then the recommended configuration would be this:
- CPU with 8 cores and 16 GB of RAM.
- Two Nvidia Titan video processors.
- Mandatory installation of 1000 GB.
- The game comes on five BluRay.
And just to buy a game was not enough, you still had to go through the installation process! Having opened the box, the user saw 8 floppy disks (+3 for Speech Pack):
Interesting fact: note that the poster shows the release date: Christmas 1991. The game was only completed in 1993 after a long process that Chris Robert called "The Apocalypse of Today's Computer Games."
Unpacking the game from floppies to the hard drive and changing 13 disks took a good half an hour. And when you already thought that it was over, the game began to generate all the cards. The size of the game increased from 24 MB to 38 MB: four times more than any other game of that time.
An interesting fact: the card was generated from one seed (integer) inserted into the pseudo-random number generation algorithm. It was a skillful technique that avoided increasing the amount of data on disks by generating a card after installation. If you're interested in the details of this trick, I recommend reading The Backroom Boys and the chapter on Frontier Elite.
On the 386th PC, card generation took ONE HOUR. But Origin Systems came up with the idea of supplying with the game box an excellent one-page magazine in the gloomy surroundings of the 2012 game world. Thanks to Sudden Death, the whole process became less painful (note that on page 38 there is a fake ad promising to release Strike Commander by Christmas 2013).
Strike Commander!
Finally, after these steps, players could enjoy the game ... in April 1993! Despite the fact that the game was delayed for two years, it had everything that Origin promised, and it sold very well. Many spent long nights fighting in battles, even though the game should have looked like this:
... at the minimum settings it looked like this:
In general, this was enough to attract pilots and create good memories.
Lost source code
After the release of Oculus Rift, my interest in Strike Commander increased again: a suitable hardware appeared. Since the game was 20 years old, I expected that the source code was already released, but quickly came across a story about the sad history of Origin Systems withering.
In September 1992, Origin Systems was bought by Electronic Arts and in about 1999 all projects were canceled due to poor sales of Ultima 9. The company had to focus on another area in which it was also the pioneer: MMORPG Ultima Online . Many people believe that the source codes and versions on the "gold" of all finished games are stored somewhere in the depths of the EA repository. But after contacting people from Wing Commander CIC , I found out that all the source code disappeared after the company was closed.
Today it’s hard to believe, but at that time developers and companies were more interested in new games, rather than preserving the “old stuff”, there wasn’t nostalgia factor yet, there wasn’t a big base of fans and shops like modern Good Old Games . Many stories can illustrate the level of “primitiveness” of source control, but perhaps the best story is “ZAP SC” at 15 minutes 14 seconds:
On the first day of work, one developer managed to delete the entire 900 MB Strike Commander source tree. The IT department spent 72 hours restoring everything from the machines of the developers. The interview also mentions that the Wing Commander 1 and 2 code was transferred on floppy disks: before Strike Commander they had no network!
Interesting fact: an unexpected turn of events - part of the source code was recently found by former Origin developers: Wing Commander CIC stores an offline archive that stores the source codes of Wing Commander I and Wing Commander III . People who worked on Ultima 8, the anonymous source was proposed (but rejected) the source code «Ultima 8: Pagan». As for the Strike Commander, I never heard anyone stay with him.
Reverse engineering: is it possible?
Over the years, many teams managed to work on Origin Systems games and achieved excellent results:
- In Underworld II Revial recreated the Ultima Underworld II of: Labyrinth of Worlds .
- In Exult recreated the Ultima VII of: Glack of The Gate .
- The Pentagram recreated the Ultima VIII of: Pagan (I advise you to see the code design based on the actors, is the process inside the nucleus, is fine).
How long and hard can it be? After talking with Gregory Montoir, who did almost the entire reverse development of Another World , I realized that working an hour in the evening can convert 10 KB of assembler code back to C in a month. Strike Commander has a lot of executable files, and the initial discouraging estimate of the time was this: It was frustrating: if someone started work in 1993, they would have finished six years ago, and it would be enough for me to do it ! But by running the IDA and examining the files a bit, I realized that not everything needs to be reversed: the 3D engine is completely in STRIKE.EXE and a small team will be able to handle it in a reasonable amount of time.
INSTALL.EXE 7 793 байт : 2 недели
MKTERR.EXE 203 744 байт : 1,5 года
SC.EXE 20 000 байт : 1 месяц
MKGAME.EXE 131 696 байт : 1 год
OPTTEST.EXE 870 528 байт : 7 лет
STRIKE.EXE 746 304 байт : 6 лет
=============================================
15 лет, 7 месяцев и 2 недели. Ой-ёй.git cloneRoad map
The roadmap I originally drew for Strike Commander's reverse engineering was this:
- Collect as much documentation as possible.
- Understand the global architecture of Strike Commander.
- Reverse game resource development.
- Document stage 3 and extrapolate (Visual Surface Determination, Level Of Detail, etc.)
- Re-implement the 3D engine and assemble
NEO_STRIKE.EXE - Add support for the Oculus Rift VR device.
And I got to work.
Part 2. Architecture and documentation,
Architecture
Strike Commander does not consist of a single monolithic executable. Six executable files work together to provide gameplay. The basic idea is similar to the Second Reality code review . The use of multiple .exe files can be explained in different ways:
- Optimization of collaboration in a team (each employee could work on his part without affecting others).
- Because of the real DOS mode, programs were limited to 640 kilobytes of RAM. A large monolithic executable file would lead to an increase in data exchange with the disk or it would not load at all.
After studying in the IDA, it turned out that each executable file uses the DOS 21h system call to load and run others
.EXE. For example, the story mode, which provides the processing of dialogs and cinematic inserts - this OPTTEST.EXE. It removes itself from RAM and loads / starts STRIKE.EXEwhen 3D mode is needed. For research and study of the components responsible for each part of Strike Commander, DosBOX proved to be very useful. The name of the currently running executable is shown in the window title:
| Executable file | Screenshot | Notes |
|---|---|---|
| INSTALL.EXE |  | Module starting MKGAME.EXE |
| MKGAME.EXE |   | Real installer:
|
| MKTERR.EXE |    | A map generator that creates all the map elements and groups them in PAK:
|
| OPTTEST.EXE |    | He is responsible for all in-game dialogs, cinematic inserts, menus, the choice of weapons in the hangar and viewing objects. |
| SC.EXE | | The way the game starts. Usually it loads and executes OPTTEST.EXE. |
| STRIKE.EXE |  | Three-dimensional engine RealSpace. Responsible for the active phase of the gameplay. |
Note: if you delete the PAK card archive, it
STRIKE.EXEdetects the absence of a file and automatically starts MKTERR.EXEto generate a card. That is, most of the work happens in OPTTEST.EXEand STRIKE.EXE. But we have not yet figured out how the parameters are passed between them. The DOS 21h system call allows you to use command line parameters, store data in a specific memory page, and the game state can also be saved on your hard drive. The IDA will tell us how things are in our case.List of important files
Important files used by the game:
//Исполняемые файлы
16 -rw-r--r-- 1 fabiensanglard staff 7,793 17 Jan 03:02 INSTALL.EXE
264 -rw-r--r-- 1 fabiensanglard staff 131,696 17 Jan 03:02 MKGAME.EXE
400 -rw-r--r-- 1 fabiensanglard staff 203,744 17 Jan 03:03 MKTERR.EXE
1704 -rw-r--r-- 1 fabiensanglard staff 870,528 17 Jan 03:02 OPTTEST.EXE
16 -rw-r--r-- 1 fabiensanglard staff 7,793 17 Jan 03:09 SC.EXE
1464 -rw-r--r-- 1 fabiensanglard staff 746,304 17 Jan 03:03 STRIKE.EXE
//Ресурсы
19832 -rw-r--r-- 1 fabiensanglard staff 10,150,560 17 Jan 03:03 GAMEFLOW.TRE
952 -rw-r--r-- 1 fabiensanglard staff 485,877 17 Jan 03:02 MISC.TRE
1304 -rw-r--r-- 1 fabiensanglard staff 665,456 17 Jan 03:02 MISSIONS.TRE
13544 -rw-r--r-- 1 fabiensanglard staff 6,932,708 17 Jan 03:02 OBJECTS.TRE
1760 -rw-r--r-- 1 fabiensanglard staff 899,145 17 Jan 03:02 SOUND.TRE
3288 -rw-r--r-- 1 fabiensanglard staff 1,681,738 17 Jan 03:02 TEXTURES.TRE
//Карты
2040 -rw-r--r-- 1 fabiensanglard staff 1,042,674 17 Jan 03:05 ALASKA.PAK
2040 -rw-r--r-- 1 fabiensanglard staff 1,042,570 17 Jan 03:04 ANDMAL1.PAK
2040 -rw-r--r-- 1 fabiensanglard staff 1,042,960 17 Jan 03:09 ANDMAL2.PAK
2048 -rw-r--r-- 1 fabiensanglard staff 1,046,382 17 Jan 03:09 ARENA.PAK
2040 -rw-r--r-- 1 fabiensanglard staff 1,043,268 17 Jan 03:06 CANYON.PAK
2032 -rw-r--r-- 1 fabiensanglard staff 1,038,716 17 Jan 03:05 EGYPT.PAK
2024 -rw-r--r-- 1 fabiensanglard staff 1,033,096 17 Jan 03:07 EUROPE.PAK
656 -rw-r--r-- 1 fabiensanglard staff 333,464 17 Jan 03:02 MAPDATA.PAK
2040 -rw-r--r-- 1 fabiensanglard staff 1,044,396 17 Jan 03:03 MAURITAN.PAK
2032 -rw-r--r-- 1 fabiensanglard staff 1,037,798 17 Jan 03:04 QUEBEC.PAK
2040 -rw-r--r-- 1 fabiensanglard staff 1,043,840 17 Jan 03:08 RHODEI.PAK
2048 -rw-r--r-- 1 fabiensanglard staff 1,046,316 17 Jan 03:06 SANFRAN.PAK
2048 -rw-r--r-- 1 fabiensanglard staff 1,045,766 17 Jan 03:08 TURKEY.PAK
//Файлы начальных чисел для генерирования карт
80 -rw-r--r-- 1 fabiensanglard staff 37,732 17 Jan 03:02 MSFILES.PAK
// Палитра 3D-движка
8 -rw-r--r-- 1 fabiensanglard staff 1,806 17 Jan 03:02 PALETTE.IFFDocumentation Related to Strike Commander
- PAK archive format ( wc1g.txt ).
- TRE archive format ( wc1g.txt ).
- RLE, Run Length Encoding for images and animations ( wc1g.txt ).
- Формат IFF (IFF.txt и A Quick Introduction to IFF.txt).
- Аудиоформат Creative Voice (VOC) (Creative Voice (VOC) file format.txt).
- Формат музыки Extended MIDI (XMIDI). Исходный код формата открыт его автором Джоном Майлсом (AIL2.ZIP и XMIDI.TXT).
- Система разработки Borland DOS C++ Development System (Borland C++ Power Programming Book and Disk Programming_.pdf).
- The Art of Assembly Language DOS 16bits edition (веб-сайт).
- Бесплатная версия IDA (загрузка IDA5.0).
- Руководство для плейтестеров Origin PlayerTester Guide (содержит подробности об области повреждений в самолёте) Strike Commander Playtesters Guide (1998)(Origin Systems).pdf.
- Аэродинамические свойства реактивного самолёта, Уэйн Сайкс, Game Developer Magazine, декабрь 1994 года (GDM_December_1994.pdf).
- Журнал Sudden Death (StrikeCommanderManual.pdf).
- Руководство по стратегиии в Strike Commander (StrikeCommanderStrategyGuide.pdf).
- Технический отчёт (Origin_Software_Technogoly_Entertainment_Report.pdf).
- Брайан Хук, «Создание трёхмерного игрового движка на C++»: программирование драйверов Thrust Master (AST3D.zip).
Часть 3. Обратная разработка ресурсов.
Archive system
The PAK and TRE formats are 100% clear thanks to Mario's specifications .
Archives are easy to manage / unpack with TreArchive and PakArchvive in libRealSpace.
Aircraft
Aircraft are stored in complete files in IFF archives. They are 95% understandable. Consist of four key parts:
- Aerodynamic properties.
- Damage System and Life Points
- Weapon system
- Appearance
- 3D data
- Textures
See JET format specifications .
Cards
Maps are 95% understandable. They are stored in
X.PAK, but also refer to textures in TEXTURES.TRE. See card format specifications . See map texture format specifications .
Menu
Menus are 50% understandable (visualization, but not logic).
No specifications yet.
Animations
Animations are 50% clear:
- It is possible to render and play individual layers.
- The choice of the used palette is clear.
- The collector joining the layers is unknown.
See animation format specifications .
Palettes
Palettes are 100% clear: see palette format specifications .
AI / game logic
So far they have not worked on them. They are probably all in exe.
Given the time and documentation found in the playtester’s manual, I think that there’s not a simple decision tree. Probably hierarchical finite state machines.
See a detailed discussion of all the techniques in Arctificial Intelligence for Games .
Save games
The format of saved games is 75% understandable: see the specifications for the format of saved games .
Part 4. We start the game again.
The Strike Commander rewriting project has begun. Full source code can be found on GitHub :
git clone https://github.com/fabiensanglard/libRealSpaceEngine
Aircraft / Objects
Menu
Map viewer
Graphics Format
Animation Viewer
Game Viewer
Nothing yet! I strongly hoped that the game uses a SCUMM -style virtual machine . This part is performed
OPTTEST.EXEPalettes
The effects of the palette will probably be difficult to recreate: when the pilot experiences too much overload, the screen turns gray (and then dark if it gets even larger G). This is implemented by interpolating between palettes. A shader can help here.
Oculus Rift Support
Work on the integration of Oculus Rift has not yet begun. [approx. lane: the last commit on Github of the project is dated October last year, so it seems, alas ...]