Binary format reverse engineering using Korg .SNG files as an example

We live in an amazing time. Around us is an abundance of technology: telephones, computers, smart watches and other gadgets. Every day, manufacturers are releasing more and more devices to the market. Most of them are destined for a short and bright (or not so) life: a powerful marketing company at the time of release, 1-2 years of full support by the manufacturer, and then slow oblivion. Simple devices can work for years after the end of the official support period. With smart devices, it’s getting harder. It’s good if the gadget at least continues to work after disconnecting the manufacturer’s servers / services. And it’s lucky if the next update of the OS, drivers or other software does not beat compatibility.

Unfortunately, events are increasingly developing according to a pessimistic scenario. And 5-10 years after the purchase, we have technically sound devices in our hands, which nevertheless cannot be used due to the lack of software support. Of course, a broken gadget is unpleasant. But much more unpleasant if there is user data in an incompatible format with anything. This data can be considered lost if the device stops functioning. In my case, the worst has not happened yet, but the alarm bells are already ringing.

So, there is the notorious company Korg, which produces very high-quality musical equipment. In 2010, I bought a synthesizer from this company to practice music as a hobby. Korg microstation is a fairly advanced model. Among other things, it has a sequencer on board for recording its tracks and can write data to a memory card in the proprietary SNG format. It is possible to export to a common midi format, but almost all metadata is lost: information about superimposed effects and filters, various settings of virtual instruments, etc. The main problem for me personally is the speed of transition to recording musical ideas. The muse is a whimsical creation, and most of the time I come across an interesting idea simply by improvising or playing something uncomplicated. The faster I press the record button without wandering around the menu, the more likely I can repeat and record an interesting fragment, which in the future may become part of a full-fledged work. Of course, this approach is imperfect, but we are talking about a hobby. One way or another, for almost ten years, I have accumulated about a thousand musical sketches and sketches in SNG format.

The bell rang in the form of a series of glitches of the synthesizer, requiring a flashing of the device. And I thought about converting all my accumulated data into the Midi format, all the more so since it will make it much easier to store, organize and edit them. The search for the converter in Google did not give anything. There are many requests on all kinds of forums, the history has been going on for 20 years, if not more. All I found was someone else's ancient Windows utility, naturally incompatible with my files.

And then I decided to try to see what this SNG format is all about? Maybe somewhere inside there are normal MIDI data that you can easily pull out and save?

An attempt to solve the problem "forehead"

So, from the instructions for the synthesizer, you can find out that the SNG format is a container in which the so-called "songs" are stored. Each song contains 16 sequencer tracks with music data, as well as settings for sounds and effects. When exporting to Midi format via the synthesizer menu, each “song” is exported to a separate .MID file, and all settings for sounds and effects are lost. Because I play my ideas in the simplest form and without effects, the problem is precisely a large number of SNG files and the inconvenience of the manual conversion process. Let's see if this process can be accelerated or automated.

First, let's recall what MIDI data is. Simply put, this is a stream of musical events: pressing and releasing a key, pressing and releasing a sustain pedal, changing the tempo, patch (virtual instrument) and other parameters. Each event contains a time delta from the moment of the previous event and data, for example, note intensity and pitch. The midi file format is very simple: in addition to the headers and the data itself, there is practically nothing there.

Pink is a Note On event. Pale yellow is a delta of time. Blue - Note Off event.

Let's try to look for our midi data in the SNG file. To do this, write a sequence of several notes on the synthesizer and export it to both formats. Because Since we don’t know where exactly the musical data is in the binary files, we’ll try to repeat the process with different sequences of notes.

Hereinafter I use the Hex-editor Synalyze It! Its capabilities in the future will be very useful to us. In the meantime, just use the binary comparison function.

In fact, only the name of the “song” coincided. Comparing two SNG files with different sequences of notes, we can roughly guess where exactly the musical data is stored, but for now this will not help us in any way - the data format is different. The file itself is ten times larger than the Midi file and seems to contain a lot of additional information. You can see the KORG signature in the first four bytes and some other lines, including the name of the “song” and the names of the patches (tones) assigned to the tracks.

Parsing the structure of data blocks

This could be completed if, fortunately, there were no tools that made it relatively easy to analyze and understand the structure of binary data. The same program Synalaze It! Will help us with this, which allows you to create and apply a “grammar” for analyzing binary files.

Grammar is a hierarchical descriptive structure that allows you to represent binary data in a human-readable form. The program allows you to download grammar for some formats. For example, for the same midi:

For the SNG format, no ready-made grammar was expectedly found. Well, let's see what we can extract from the file on our own.

Let's start with the headline. Typically, this part contains the file signature, version information, sizes and offsets of data blocks. After comparing several different SNG files, we find the invariable parts and pay closer attention to those that change

Create a title structure in the grammar editor. The first 4 bytes are obviously the signature of the file. Assume the following 4 bytes are versioned. The next few tens of bytes do not change and do not contain anything interesting - we will create for them binaryData of the appropriate size. But then the fun begins. You can notice some patterns in the behavior of bytes at offsets 0x13 and 0x1b. The second seems to correspond to the number of “songs” in our file. And the first one also grows with the amount of data in the header - this seems to be the size, only the countdown does not come from the beginning of the file, but from the next byte 0x14. At this stage, we can only guess about the type of numerical data. Suppose the size is of type UInt32, i.e. takes 4 bytes. Add them to our structure. Now we can set the size of the header structure (size + 20).


Let's see what comes next. If you look closely, you will notice that three-letter abbreviations are scattered throughout the file: SNG1, SDK1, SGS1, and so on. These characters are found in all SNG files, so we can assume that these are signatures of certain blocks. In addition, our title ended very successfully just before one of these signatures. Compare the behavior of the following 4 bytes in files of different sizes. It can be seen that the values ​​increase with increasing amount of data.

A few more experiments, analysis and calculations, and the following picture begins to emerge:




Thus, our file consists of a fairly simple hierarchy of blocks. There are parent blocks that can contain multiple child blocks. There are leaf blocks (in the terminology of binary trees) that do not contain other blocks.

Then the magic begins. With just a few grammar structures, we can completely parse the block structure of the file.

So, create a DataChunk template structure with the following fields (the size in square brackets):

id: String [4]
size: Int [4]
hierarchy: Int [4]
data: structure

Now create a parentChunk structure that inherits from DataChunk. In the hierarchy property, specify Fixed Value 0x400 - this is a sign of the parent block. Be sure to check the Must match checkbox.

Similarly, create childChunk. Hierarchy in this case will have two values: 0x240100 and 0x100

Add links to the parentChunk and childChunk structures to the data structure of parentChunk - this way we create recursion.

Finally, add a reference to the parentChunk structure in the main node.

The order of elements in the data structure of parentChunk must be Variable, it is also required to set the minimum and maximum number of children of this structure: 0 and Unlimited, respectively.

We’ll apply the changes, and voila - our file is nicely parsed into the main blocks.

We still don’t know anything about the data itself, but now we can easily navigate in the file and focus on finding the information we need.

Parsing a block containing a table of contents of a file

For training, let's try to parse some simple block, for example, SDK1. Apparently, it contains something like a table of contents - a list of songs and probably some offsets / sizes.

Create an sdk1Chunk structure that inherits childChunk. We will edit the ID field, indicating the signature of our block in the Fixed Values ​​field. Do not forget about the Must match checkbox. In the block data, one can observe a fairly obvious repeating pattern: the name of the “song” and so far unknown data. Note that the size of the repeating fragments is 64 bytes. Also, comparing the file versions with a different number of “songs”, you can determine that the number is stored in the first four bytes. Using simple calculations and making several assumptions, we obtain the following version of the structure in the grammar:

Here I created a 64 byte songInfo child structure and indicated the ability to repeat numSongs times. This is how the result of applying the grammar looks:

Further analysis of the file remains a matter of technique. I changed the general settings of the “song” and the parameters of individual tracks on the synthesizer. By comparing file versions with various changes, you can improve and refine the grammar. After a sufficiently large number of such iterations, there are almost no unrecognized pieces of data in the file. I got a little carried away by the process and sorted out almost all sections of the file, although this was not required for the original task.


I will miss the details of this process - in the future we will focus on the analysis of the musical data directly.

But more about that in the next part. There we will encounter an interesting task of data conversion (it is quite suitable for interviews), try to solve it with a small script and hear a rather unusual test conversion result.

Preliminary Results

The need for reverse engineering binary files may occur unexpectedly. For example, to analyze device firmware, convert from rare data formats, analyze digital threats, or even trivially modify game saves. Modern tools allow you to solve these problems quickly and efficiently. About 10 years ago, I was researching laptop firmware and this process could take several weeks. Then it was required to manually write scripts to analyze data blocks and lay out structures. With a new partially automated approach, I created an almost complete file grammar in just a couple of days.

You can start the analysis of the binary file by searching for strings - they can give the first clues and speed up the analysis process. Often binary files consist of data blocks that are organized in a hierarchical or linear structure. If you deal with this structure, then further analysis will be much easier. The file header can give hints on the offsets / sizes of the data blocks. In the first stages, it makes sense to focus on the description of obvious structures and blocks. The analysis task is greatly simplified by the ability to create new versions of files with different settings, parameters, data. There are a number of difficulties associated with unknown data types and byte order in their binary representation (Endianness). We will touch upon these questions in the next part.

Recommended Reading

Andreas Pehnack. How to Approach Binary File Format Analysis