Back Engineering Caesar III

    I like to play games, especially economic strategies, I want to talk about the urban development simulator from childhood - Caesar III, as they say, warm and lamp. The game was released in 1998 by experts in their field, Impressions Games. This is an economic simulator of managing an ancient Roman city in real time. After many years, I decided to go through it again, and then try to extend the pleasure of the game, look at the resources and understand the game logic from the point of view of the programmer.

    Under the cut, I will describe the process of extracting textures, searching for game algorithms and telling how a hobby has turned into an independent project. And there will also be a RGB555 palette, IDA, HexRays and some code.


    Music
    I won’t write anything about music, because it lies unpacked by anyone on a disc with a game in .wav format.

    Graphics

    With graphics (textures), everything is much more complicated, the textures are divided into several pseudo archives with the extension .sg2 and .555 .

    A file with the extension .sg2 , let's call it the “table of contents”, contains texture parameters: dimensions, offset in the atlas, name and number, identifier, various flags.

    A file with the extension .555 , let's call it “atlas”, contains the images themselves, in their own description format, which are divided into three types:
    - simple (bmp)
    - isometric
    - with alpha channel
    Each texture type has its own “compression” format. The “table of contents” can refer to several atlases, and the name of the “atlas” must correspond to the name of the group of textures that it contains. Simple textures are read as an array of colors and they can be drawn practically without any processing on the screen; the “processing” consists in converting BGR555 colors with a depth of 5 bits per channel to a more convenient ARGB32 for work. The Caesar III game does not use textures with transparency, they will be used later in this series of games (Pharaoh, Cleopatra, etc.).

    The C3.SG2 file contains descriptions of image groups.
    If you open this file in a hex editor, you can see the following data block

    that describes a group of 44 (n_images: 0x0000002C) images with the nameplateau , information about which starts with index 201 (start_index: 0x000000C9). In total there is room for 100 such groups in the “table of contents”. After the description of the groups, there are descriptions of specific images, through which you can restore the pictures themselves. The only thing left is to read the table of contents, unpack the squeezed textures and assemble them into full-fledged images. Here's what happened when unpacking the plateau group.


    Here are some more restored textures, in the native format, as far as it turned out, without filters.


    And here are the processed textures with alpha channel.


    If you can still figure out the atlas of textures and the data structures used in it, relying on ingenuity, a hex editor and a bit of luck, then this will not work with texture restoration algorithms. And here Ilfak comes to the rescue with the indispensable IDA debugger, and the equally useful Hex-Rays decompiler. We open c3.exe in the debugger, we see a picture that is by no means a rainbow, most of the time I program in Java (java) or pluses (c ++) and for me this is not like a dark forest, but a dense bush for sure.


    Here, the ability of the IDA to restore asm to the plain-C pseudo-code will help us. We press F5 and we have a human-readable code, which you can already work with.
    .

    With functions and variables, and a structured structure, and probably the discerning reader noticed some regularity in the above code, so let's make it more readable. Press the N button , enter the normal name for the function, and the code looks much simpler.


    And after some time (day, week, month, etc.) it will become like this. Agree, now it is much more convenient to search for algorithms
    .

    The Caesar III game executable file was compiled with debugging information by the Visual C ++ 5.0 compiler, which also allows you to restore application logic more efficiently. Using the debugger, decompiler and your own gray cells, you can get to the function of reading images from the archive
    A lot of code
    int __cdecl fun_drawGraphic(signed int graphicId, int xOffset, int yOffset)
    {
      int result; // eax@2
      LONG v4; // [sp+50h] [bp-8h]@43
      drawGraphic_graphicId = graphicId;
      drawGraphic_xOffset = xOffset;
      drawGraphic_yOffset = yOffset;
      if ( graphicId <= 0 )
        return 0;
      if ( graphicId >= 10000 )
        return 0;
      drawGraphic_fileOffset = c3_sg2[graphicId].offset;
      if ( drawGraphic_fileOffset <= 0 )
        return 0;
      LOWORD(drawGraphic_width) = c3_sg2[graphicId].width;
      LOWORD(drawGraphic_height) = c3_sg2[graphicId].height;
      drawGraphic_type = c3_sg2[graphicId].type;
      graphic_xOffset = xOffset;
      graphic_yOffset = yOffset;
      drawGraphic_visiblePixelsClipX = (signed __int16)drawGraphic_width;
      if ( c3_sg2[graphicId].extern_flag && (signed __int16)drawGraphic_width <= ddraw_width )
      {
        strcpy(drawGraphic_555file, &c3sg2_bitmaps[200 * c3_sg2[graphicId].bitmap_id]);
        j_fun_changeFileExtensionTo(drawGraphic_555file, &extension_555[4 * graphics_format_id]);
        if ( !j_fun_readDataFromFilename(
                drawGraphic_555file,
                screen_buffer,
                c3_sg2[graphicId].data_length,
                c3_sg2[graphicId].offset - 1) )
        {
          j_fun_changeFileExtensionTo(drawGraphic_555file, "555");
          if ( !j_fun_readDataFromFilename(
                  drawGraphic_555file,
                  screen_buffer,
                  c3_sg2[graphicId].data_length,
                  c3_sg2[graphicId].offset - 1) )
            return 0;
          if ( c3_sg2[graphicId].compr_flag )
            j_fun_convertCompressedGraphicToSurfaceFormat(screen_buffer, c3_sg2[graphicId].data_length);
          else
            j_fun_convertGraphicToSurfaceFormat(screen_buffer, c3_sg2[graphicId].data_length);
        }
        j_fun_setGraphicXClipCode();
        j_fun_setGraphicYClipCode();
        if ( drawGraphic_clipYCode == 5 )
          return 0;
        if ( drawGraphic_type )
        {
          if ( drawGraphic_clipYCode == 5 )
            return 0;
          drawGraphic_fileOffset = 2 * (signed __int16)drawGraphic_width * drawGraphic_invisibleHeightClipTop;
          drawGraphic_fileOffset += 2 * drawGraphic_invisibleWidthClipLeft;
          if ( drawGraphic_clipXCode == 1 )
          {
            j_fun_drawGraphicUncompressedClipLeft((char *)screen_buffer + drawGraphic_fileOffset);
          }
          else
          {
            if ( drawGraphic_clipXCode == 2 )
              j_fun_drawGraphicUncompressedClipRight((char *)screen_buffer + drawGraphic_fileOffset);
            else
              j_fun_drawGraphicUncompressedClipY((char *)screen_buffer + drawGraphic_fileOffset);
          }
        }
        else
        {
          if ( c3_sg2[graphicId].compr_flag )
          {
            if ( drawGraphic_clipXCode == 1 )
            {
              j_fun_drawGraphicCompressedClipLeft((char *)screen_buffer);
            }
            else
            {
              if ( drawGraphic_clipXCode == 2 )
                j_fun_drawGraphicCompressedClipRight((char *)screen_buffer);
              else
                j_fun_drawGraphicCompressedFull((char *)screen_buffer);
            }
          }
          else
          {
            drawGraphic_fileOffset = 2 * (signed __int16)drawGraphic_width * drawGraphic_invisibleHeightClipTop;
            drawGraphic_fileOffset += 2 * drawGraphic_invisibleWidthClipLeft;
            if ( drawGraphic_clipXCode == 1 )
            {
              j_fun_drawGraphicUncompressedClipLeft((char *)screen_buffer + drawGraphic_fileOffset);
            }
            else
            {
              if ( drawGraphic_clipXCode == 2 )
                j_fun_drawGraphicUncompressedClipRight((char *)screen_buffer + drawGraphic_fileOffset);
              else
                j_fun_drawGraphicUncompressedClipY((char *)screen_buffer + drawGraphic_fileOffset);
            }
          }
        }
        result = (signed __int16)drawGraphic_width;
      }
      else
      {
        if ( c3_sg2[graphicId].extern_flag )
        {
          if ( window_id == 21 || window_id == 20 )
          {
            drawGraphic_visiblePixelsClipX = fullscreenImage_width;
            drawGraphic_visiblePixelsClipY = fullscreenImage_height;
            drawGraphic_copyBytesInBufferForClipX = 2 * ((signed __int16)drawGraphic_width - drawGraphic_visiblePixelsClipX);
            drawGraphic_skipBytesInBufferForClipX = 2 * (ddraw_width - drawGraphic_visiblePixelsClipX);
            j_fun_drawGraphicUncompressedFull(&c3_555[2 * fullscreenImage_xOffset + 13000000] + 2
                                                                                              * (signed __int16)drawGraphic_width
                                                                                              * fullscreenImage_yOffset);
            return drawGraphic_visiblePixelsClipX;
          }
          v4 = 2 * (signed __int16)drawGraphic_width * fullscreenImage_yOffset + 2 * fullscreenImage_xOffset;
          drawGraphic_visiblePixelsClipX = fullscreenImage_width;
          drawGraphic_visiblePixelsClipY = fullscreenImage_height;
          strcpy(drawGraphic_555file, &c3sg2_bitmaps[200 * c3_sg2[graphicId].bitmap_id]);
          j_fun_changeFileExtensionTo(drawGraphic_555file, &extension_555[4 * graphics_format_id]);
          if ( !j_fun_readUncompressedImageData(
                  drawGraphic_555file,
                  screen_buffer,
                  2 * drawGraphic_visiblePixelsClipX,
                  drawGraphic_visiblePixelsClipY,
                  v4) )
          {
            j_fun_changeFileExtensionTo(drawGraphic_555file, "555");
            if ( !j_fun_readUncompressedImageData(
                    drawGraphic_555file,
                    screen_buffer,
                    2 * drawGraphic_visiblePixelsClipX,
                    drawGraphic_visiblePixelsClipY,
                    v4) )
              return 0;
            j_fun_convertGraphicToSurfaceFormat(
              screen_buffer,
              drawGraphic_visiblePixelsClipY * 2 * drawGraphic_visiblePixelsClipX);
          }
          drawGraphic_copyBytesInBufferForClipX = 0;
          drawGraphic_skipBytesInBufferForClipX = 0;
          j_fun_drawGraphicUncompressedFull((char *)screen_buffer);
          result = drawGraphic_visiblePixelsClipX;
        }
        else                                        // internal
        {
          if ( (unsigned __int8)drawGraphic_type == 30 )// isometric
          {
            switch ( (signed __int16)drawGraphic_width )
            {
              case 58:
                LOWORD(drawGraphic_height) = 30;
                break;
              case 26:
                LOWORD(drawGraphic_height) = 14;
                break;
              case 10:
                LOWORD(drawGraphic_height) = 6;
                break;
              default:
                if ( (signed __int16)drawGraphic_width == 118 )
                  return j_fun_drawBuildingFootprintSize2();
                if ( (signed __int16)drawGraphic_width == 178 )
                  return j_fun_drawBuildingFootprintSize3();
                if ( (signed __int16)drawGraphic_width == 238 )
                  return j_fun_drawBuildingFootprintSize4();
                if ( (signed __int16)drawGraphic_width == 298 )
                  return j_fun_drawBuildingFootprintSize5();
                break;
            }
          }
          j_fun_setGraphicXClipCode();
          j_fun_setGraphicYClipCode();
          if ( drawGraphic_clipYCode == 5 )
          {
            result = 0;
          }
          else
          {
            if ( drawGraphic_type )
            {
              if ( (unsigned __int8)drawGraphic_type == 30 )
              {
                if ( drawGraphic_clipXCode == 1 )
                {
                  switch ( (signed __int16)drawGraphic_width )
                  {
                    case 58:
                      j_fun_drawBuildingFootprint_xClipRight(&c3_555[drawGraphic_fileOffset], drawGraphic_clipYCode);
                      break;
                    case 26:
                      j_fun_drawBuildingFootprint_26px_xClipRight();
                      break;
                    case 10:
                      j_fun_drawBuildingFootprint_10px_xClipRight();
                      break;
                    default:
                      j_fun_drawGraphicUncompressedClipLeft(&c3_555[drawGraphic_fileOffset]);
                      break;
                  }
                }
                else
                {
                  if ( drawGraphic_clipXCode == 2 )
                  {
                    switch ( (signed __int16)drawGraphic_width )
                    {
                      case 58:
                        j_fun_drawBuildingFootprint_xClipLeft(&c3_555[drawGraphic_fileOffset], drawGraphic_clipYCode);
                        break;
                      case 26:
                        j_fun_drawBuildingFootprint_26px_xClipLeft();
                        break;
                      case 10:
                        j_fun_drawBuildingFootprint_10px_xClipLeft();
                        break;
                      default:
                        j_fun_drawGraphicUncompressedClipRight(&c3_555[drawGraphic_fileOffset]);
                        break;
                    }
                  }
                  else
                  {
                    switch ( (signed __int16)drawGraphic_width )
                    {
                      case 58:
                        j_fun_drawBuildingFootprint_xFull(&c3_555[drawGraphic_fileOffset], drawGraphic_clipYCode);
                        break;
                      case 26:
                        j_fun_drawBuildingFootprint_26px_xFull();
                        break;
                      case 10:
                        j_fun_drawBuildingFootprint_10px_xFull();
                        break;
                      default:
                        j_fun_drawGraphicUncompressedClipY(&c3_555[drawGraphic_fileOffset]);
                        break;
                    }
                  }
                }
              }
              else
              {
                if ( (unsigned __int8)drawGraphic_type == 13 && drawGraphic_clipXCode )
                {
                  j_fun_drawImage_32x32((int *)&c3_555[drawGraphic_fileOffset]);
                }
                else
                {
                  if ( (unsigned __int8)drawGraphic_type == 12 && drawGraphic_clipXCode )
                  {
                    j_fun_drawImage_24x24((int *)&c3_555[drawGraphic_fileOffset]);
                  }
                  else
                  {
                    if ( (unsigned __int8)drawGraphic_type == 10 && drawGraphic_clipXCode )
                    {
                      j_fun_drawImage_16x16((int *)&c3_555[drawGraphic_fileOffset]);
                    }
                    else
                    {
                      if ( (unsigned __int8)drawGraphic_type == 2 && drawGraphic_clipXCode )
                      {
                        j_fun_drawGraphicType2(&c3_555[drawGraphic_fileOffset]);
                      }
                      else
                      {
                        if ( (unsigned __int8)drawGraphic_type == 20 )
                        {
                          if ( drawGraphic_clipXCode == 1 )
                          {
                            j_fun_drawGraphicLetterColoredClipLeft(&c3_555[drawGraphic_fileOffset]);
                          }
                          else
                          {
                            if ( drawGraphic_clipXCode == 2 )
                              j_fun_drawGraphicLetterColoredClipRight(&c3_555[drawGraphic_fileOffset]);
                            else
                              j_fun_drawGraphicLetterColoredFull(&c3_555[drawGraphic_fileOffset]);
                          }
                        }
                        else
                        {
                          drawGraphic_fileOffset += 2
                                                  * (signed __int16)drawGraphic_width
                                                  * drawGraphic_invisibleHeightClipTop;
                          drawGraphic_fileOffset += 2 * drawGraphic_invisibleWidthClipLeft;
                          if ( drawGraphic_clipXCode == 1 )
                          {
                            j_fun_drawGraphicUncompressedClipLeft(&c3_555[drawGraphic_fileOffset]);
                          }
                          else
                          {
                            if ( drawGraphic_clipXCode == 2 )
                            {
                              j_fun_drawGraphicUncompressedClipRight(&c3_555[drawGraphic_fileOffset]);
                            }
                            else
                            {
                              if ( drawGraphic_clipYCode )
                                j_fun_drawGraphicUncompressedClipY(&c3_555[drawGraphic_fileOffset]);
                              else
                                j_fun_drawGraphicUncompressedFull(&c3_555[drawGraphic_fileOffset]);
                            }
                          }
                        }
                      }
                    }
                  }
                }
              }
            }
            else                                    // type == 0
            {
              if ( c3_sg2[graphicId].compr_flag )
              {
                if ( drawGraphic_clipXCode == 1 )
                {
                  j_fun_drawGraphicCompressedClipLeft(&c3_555[drawGraphic_fileOffset]);
                }
                else
                {
                  if ( drawGraphic_clipXCode == 2 )
                    j_fun_drawGraphicCompressedClipRight(&c3_555[drawGraphic_fileOffset]);
                  else
                    j_fun_drawGraphicCompressedFull(&c3_555[drawGraphic_fileOffset]);
                }
                if ( drawGraphic_colorMask )
                {
                  if ( drawGraphic_clipXCode == 1 )
                  {
                    j_fun_drawGraphicCompressedColorMaskClipLeft(&c3_555[drawGraphic_fileOffset], drawGraphic_colorMask);
                  }
                  else
                  {
                    if ( drawGraphic_clipXCode == 2 )
                      j_fun_drawGraphicCompressedColorMaskClipRight(&c3_555[drawGraphic_fileOffset], drawGraphic_colorMask);
                    else
                      j_fun_drawGraphicCompressedColorMaskFull(&c3_555[drawGraphic_fileOffset], drawGraphic_colorMask);
                  }
                }
              }
              else                                  // not compressed
              {
                drawGraphic_fileOffset += 2 * (signed __int16)drawGraphic_width * drawGraphic_invisibleHeightClipTop;
                drawGraphic_fileOffset += 2 * drawGraphic_invisibleWidthClipLeft;
                if ( drawGraphic_clipXCode == 1 )
                {
                  j_fun_drawGraphicUncompressedClipLeft(&c3_555[drawGraphic_fileOffset]);
                }
                else
                {
                  if ( drawGraphic_clipXCode == 2 )
                    j_fun_drawGraphicUncompressedClipRight(&c3_555[drawGraphic_fileOffset]);
                  else
                    j_fun_drawGraphicUncompressedClipY(&c3_555[drawGraphic_fileOffset]);
                }
              }
            }
            result = drawGraphic_visiblePixelsClipX;
          }
        }
      }
      return result;
    }
    


    Based on this code, it will be possible to build an application that can display the textures used in the game.

    Hobbies
    It would be strange if the post about back-engineering of the game ended with a link to someone else's program))) My passion for modding the game’s resources grew first into writing a number of fixes that fix some errors, and now into a full-fledged remake of the game .
    If you are not very interested in reading thoughts about the goals of the project and copyrights, you can go to the download section and see how I managed to get closer to the original.

    What are the goals of the remake
    + Allow other people to play a forgotten game and not only under Windows.
    + Play Caesar III without emulators, dances with a tambourine, fuss with the launch of the game under Wine, currently wild resolution of 800x600.
    + Improve the quality of textures, fonts and game speed.
    + Enjoy the development - I like to play games, especially economic ones, and I really don’t like when the game is buggy, crashes or does not work correctly. It’s easier for me to make a remake than to write my own game, because I am very critical of my programs, trying to remove glitches and adjust the balance to the maximum. But the result is always a little worse than you expect, which is probably why it takes several times more time to create your own project.
    + Finally add the network game that I so lacked in my childhood.
    + On the tablet, beat the barbarians, standing in a traffic jam - agree much more interesting than donating to the farm.
    + Make a good translation, not only for Russian speakers, but for the French, for example, the game reached them in English.

    What to do with copyright
    There are few options:
    1. To hammer and do what you want is not our way, we are civilized people, I don’t want to spend a huge amount of time on a remake, so that the authors of the original forbid it to finish.
    2. Write to the copyright holder by mail and ask for permission (verbal, permission to use resources or a brand, “on paper”, etc.). It is even worse, civilized authors, or rights holders (at the moment this is Activision), as a rule, hold on to them until the last, even if the game does not make a profit. There are rights - then there will be no remake. Point.
    3. Position the game as a mod that needs an original game to work,downloaded from torrent honestly bought on GOG.com, such as Corsix TH, for example, releasing a remake of Theme Hospital. The most justified and safest way, though ...

    Old games don't mean bad. Many old games, if you blow dust off them, clean, grease and glue ... These modern toys are stuffed into the belt for many modern crafts.
    Vadim Balashov

    Thank you for reading to the end!

    PS

    Special thanks to the people who help in the development of the remake.
    Bianca van Schaik (http://pecunia.nerdcamp.net/), back-engineering of the original
    Gregoire Athanase game (http://sourceforge.net/projects/opencaesar3/), author of the render and many algorithms
    George Gaal (https: // github.com/gecube/opencaesar3) back-engineering of saves
    and many other committers


    UPD1. If you are interested in the back-engineering results of this game (exe + idb), it is better to contact via mail or PM, the topic is called “gray legal area”. To familiarize with the game, IDA 5.5 + Hex-Rays 1.01 was used. Files and materials posted with permission from Bianca van Schaik (http://caesar.biancavanschaik.nl/).

    UPD2. Why did this post get into the linux hub. OllyDbg u IDA are running on Win7 virtual machine, QtCreator 3.0.1 + cmake + gcc 4.8 is used for development, the game is natively written for linux. For assembly under Windows, the mingw-w64 cross-compiler is used; for MacOSX and Haiku, virtual machines are raised. To build for android, use the environment from libsdl-android.

    Also popular now: