Work with the raster at a low level for beginners

The reason for this article was the following post: "Converting bmp images into a matrix and vice versa for further processing . " At one time, I had to write a lot of research code in C #, which implemented various compression and processing algorithms. The fact that the code is exploratory, I mentioned is not accidental. This code has peculiar requirements. On the one hand, optimization is not very important - it is important to test the idea. Although I want this check not to stretch for hours and days (when launching with various parameters is in progress, or a large body of test images is being processed). The method used in the aforementioned post to refer to the brightness of pixels bmp.GetPixel (x, y) is where my first project started. This is the slowest, albeit the easiest way. Is it worth it to bother here? Let's measure it.

We will use the classic Bitmap (System.Drawing.Bitmap). This class is convenient because it hides from us the details of encoding raster formats - as a rule, they do not interest us. At the same time, all common formats are supported, such as BMP, GIF, JPEG, PNG.

By the way, I will offer the first benefit for beginners. The Bitmap class has a constructor that allows you to open a picture file. But it has an unpleasant feature - it leaves open access to this file, so repeated calls to it lead to execution. To fix this behavior, you can use this method, forcing the bitmap to immediately “release” the file:

public static Bitmap LoadBitmap(string fileName)
{
    using (FileStream fs = new FileStream(fileName, FileMode.Open, FileAccess.Read, FileShare.Read))
        return new Bitmap(fs);
}

Measurement Method

We will measure it by distilling into the array and back to Bitmap the classic image processing - Lena ( http://en.wikipedia.org/wiki/Lenna ). This is a free image, it can be found in a large number of image processing works (and in the title of this post too). Size - 512 * 512 pixels.

A little about the technique - in such cases, I prefer not to chase ultra-precise timers, but simply perform the same action many times. Of course, on the one hand, in this case the data and code will already be in the processor cache. But, on the other hand, we isolate the costs of the first run of the code associated with the translation of the MSIL code into the processor code and other overhead costs. To guarantee this, we first run each piece of code once - we perform the so-called “warm-up”.

We compile the code in Release. We launch it certainly not from under the studio. Moreover, it is also advisable to close the studio - it was faced with cases when the very fact of its “neglect” sometimes affects the results. Also, it is advisable to close other applications.

We run the code several times, achieving typical results - you need to make sure that it does not affect some unexpected process. Say an antivirus or something else woke up. All of these measures allow us to obtain stable, repeatable results.

The naive method

This method was used in the original article. It consists in using the Bitmap.GetPixel (x, y) method. We give the full code for a similar method that converts the contents of a bitmap into a three-dimensional byte array. In this case, the first dimension is the color component (from 0 to 2), the second is the y position, and the third is the x position. It happened in my projects, if you want to arrange the data differently - I think there will be no problems.

public static byte[, ,] BitmapToByteRgbNaive(Bitmap bmp)
{
    int width = bmp.Width,
        height = bmp.Height;
    byte[, ,] res = new byte[3, height, width];
    for (int y = 0; y < height; ++y)
    {
        for (int x = 0; x < width; ++x)
        {
            Color color = bmp.GetPixel(x, y);
            res[0, y, x] = color.R;
            res[1, y, x] = color.G;
            res[2, y, x] = color.B;
        }
    }
    return res;
}

The inverse transformation is similar, only the data transfer goes in a different direction. I will not give his code here - anyone can see the source code for the project at the link at the end of the article.

100 conversions to the image and vice versa on my laptop with an I5-2520M 2.5GHz processor, require 43.90 seconds. It turns out that with an image of 512 * 512, only about half a second is spent on data transfer!

Direct work with Bitmap data

Fortunately, the Bitmap class provides a faster way to access its data. To do this, we need to use the links provided by the BitmapData class and address arithmetic:

public unsafe static byte[, ,] BitmapToByteRgb(Bitmap bmp)
{
    int width = bmp.Width,
        height = bmp.Height;
    byte[, ,] res = new byte[3, height, width];
    BitmapData bd = bmp.LockBits(new Rectangle(0, 0, width, height), ImageLockMode.ReadOnly,
        PixelFormat.Format24bppRgb);
    try
    {
        byte* curpos;
        for (int h = 0; h < height; h++)
        {
            curpos = ((byte*)bd.Scan0) + h * bd.Stride;
            for (int w = 0; w < width; w++)
            {
                res[2, h, w] = *(curpos++);
                res[1, h, w] = *(curpos++);
                res[0, h, w] = *(curpos++);
            }
        }
    }
    finally
    {
        bmp.UnlockBits(bd);
    }
    return res;
}

This approach gives us 0.533 seconds per 100 conversions (accelerated 82 times)! I think this already answers the question - is it worth writing more complex conversion code? But can we still speed up the process by staying within the managed code?

Arrays vs pointers

Multidimensional arrays are not the fastest data structures. Here, checks are made for going beyond the limits of the index, the element itself is calculated using the operations of multiplication and addition. Since address arithmetic has already given us significant acceleration once when working with Bitmap data, maybe we will try to apply it for multidimensional arrays too? Here is the direct conversion code:

public unsafe static byte[, ,] BitmapToByteRgbQ(Bitmap bmp)
{
    int width = bmp.Width,
        height = bmp.Height;
    byte[, ,] res = new byte[3, height, width];
    BitmapData bd = bmp.LockBits(new Rectangle(0, 0, width, height), ImageLockMode.ReadOnly,
        PixelFormat.Format24bppRgb);
    try
    {
        byte* curpos;
        fixed (byte* _res = res)
        {
            byte* _r = _res, _g = _res + width*height, _b = _res + 2*width*height;
            for (int h = 0; h < height; h++)
            {
                curpos = ((byte*)bd.Scan0) + h * bd.Stride;
                for (int w = 0; w < width; w++)
                {
                    *_b = *(curpos++); ++_b;
                    *_g = *(curpos++); ++_g;
                    *_r = *(curpos++); ++_r;
                }
            }
        }
    }
    finally
    {
        bmp.UnlockBits(bd);
    }
    return res;
}

Result? 0.162 sec per 100 conversions. So they accelerated another 3.3 times (270 times compared with the "naive" version). It was such a code that I used when researching algorithms.

Why carry over?

It’s not entirely obvious why transferring data from Bitmap at all. Maybe in general, all the transformations are carried out there? I agree that this is one of the possible options. But, the fact is that many algorithms are more convenient to check with floating point data. Then there are no problems with overflows, loss of accuracy in the intermediate stages. Convert to a double / float array can be done in a similar way. Inverse conversion requires verification when converting to byte. Here is a simple code for such a check:

private static byte Limit(double x)
{
    if (x < 0)
        return 0;
    if (x > 255)
        return 255;
    return (byte)x;
}

Adding such checks and type conversions slows down our code. The version with address arithmetic on double arrays is already running 0.713 seconds (per 100 conversions). But against the background of the "naive" option - it is just lightning.

And if you need faster?

If you need faster, then we write transfer, processing in C, Asm, we use SIMD commands. Download the raster format directly, without a Bitmap wrapper. Of course, in this case we go beyond the bounds of the Managed code, with all the ensuing pros and cons. And to do this makes sense for an already debugged algorithm.

The full code for the article can be found here: rasterconversion.codeplex.com/SourceControl/latest

Update 2013-10-08:
At the suggestion of commentators, I added the option of transferring data to an array using Marshal.Copy () in the code. This is done purely for test purposes - this way of working has its limitations:

• The data order is exactly the same as in the original Bitmap. That is, the components are mixed. If we want to separate them from each other, it will still be necessary to cycle through the array, copying the data.
• The type of brightness remains byte, at the same time, it is often convenient to perform intermediate calculations with floating point.
• Marshal.Copy () works with one-dimensional arrays. Yes, they are of course the fastest and it’s not very difficult to write rgb [x + width * y] everywhere, but still ...

So, copying in two directions takes 0.158 seconds (per 100 conversions). Compared to the more flexible version on pointers, the acceleration is very small, within the statistical error of the results of different launches.

Update 2016-04-25:
User Ant00 has pointed out an error in the code of the BitmapToByteRgbQ method. It did not affect the time, but the shifting was carried out incorrectly. There was an error in copy-paste of fragments from working code. Corrected. Thank you for your perseverance (not the first time I carefully examined the code in an article that is 2.5 years old).