OutOfMemory and GDI + sometimes not at all OutOfMemory

During the last project at work, my colleague and I were faced with the fact that some methods and constructors in System.Drawing fall from OutOfMemory in completely ordinary places, and when there is a lot of free memory.

The essence of the problem

For example, take this C # code:

using System.Drawing;
using System.Drawing.Drawing2D;
namespaceTempProject {
    staticclassProgram {
        staticvoidMain() {
            var point1 = new PointF(-3.367667E-16f, 0f);
            var point2 = new PointF(3.367667E-16f, 100f);
            var brush = new LinearGradientBrush(point1, point2, Color.White, Color.Black);
        }
    }
}

When executing the last line, the OutOfMemoryException exception is guaranteed, regardless of how much free memory is available. Moreover, if you replace 3.367667E-16f and -3.367667E-16f by 0, which is very close to the truth, everything will work fine - the fill will be created. In my opinion, this behavior looks strange. Let's see why this happens and how to deal with it.

We find out the causes of the disease

Let's start by finding out what is happening in the LinearGradientBrush constructor. To do this, you can look at referencesource.microsoft.com . There will be the following:

publicLinearGradientBrush(PointF point1, PointF point2, Color color1, Color color2) {
    IntPtr brush = IntPtr.Zero;
    int status = SafeNativeMethods.Gdip.GdipCreateLineBrush(
                    new GPPOINTF(point1), 
                    new GPPOINTF(point2), 
                    color1.ToArgb(), 
                    color2.ToArgb(), 
                    (int)WrapMode.Tile, 
                    out brush
    );
    if (status != SafeNativeMethods.Gdip.Ok) 
        throw SafeNativeMethods.Gdip.StatusException(status);
    SetNativeBrushInternal(brush); 
}

It is easy to see that the most important thing here is calling the GDI + method of the GdipCreateLineBrush. So, it is necessary to watch what is happening inside it. For this we use IDA + HexRays. We load gdiplus.dll into IDA. If you need to determine which version of the library to debug, then you can use Process Explorer from SysInternals. In addition, there may be problems with the rights to the folder where gdiplus.dll lies. They are solved by changing the owner of this folder.

So, open gdiplus.dll in IDA. Wait for the file processing. After that, select in the menu: View → Open Subviews → Exports to open all the functions that are exported from this library, and find GdipCreateLineBrush there.

Thanks to character loading, HexRays power and documentation, you can easily translate the code of the method from the assembler to the readable code in C ++:

GdipCreateLineBrush

GpStatus __userpurge GdipCreateLineBrush@<eax>(int a1@<edi>, GpPointF *point1, GpPointF *point2, int color1, int color2, int wrapMode, GpRectGradient **result)
{
  GpStatus status; // esi MAPDST
  GpGradientBrush *v8; // eax
  GpRectGradient *v9; // eaxint v12; // [esp+4h] [ebp-Ch]int vColor1; // [esp+8h] [ebp-8h]int vColor2; // [esp+Ch] [ebp-4h]
  FPUStateSaver::FPUStateSaver(&v12, 1);
  EnterCriticalSection(&GdiplusStartupCriticalSection::critSec);
  if ( Globals::LibraryInitRefCount > 0 )
  {
    LeaveCriticalSection(&GdiplusStartupCriticalSection::critSec);
    if ( result && point1 && point2 && wrapMode != 4 )
    {
      vColor1 = color1;
      vColor2 = color2;
      v8 = operatornew(a1);
      status = 0;
      if ( v8 )
        v9 = GpLineGradient::GpLineGradient(v8, point1, point2, &vColor1, &vColor2, wrapMode);
      else
        v9 = 0;
      *result = v9;
      if ( !CheckValid<GpHatch>(result) )
        status = OutOfMemory;
    }
    else
    {
      status = InvalidParameter;
    }
  }
  else
  {
    LeaveCriticalSection(&GdiplusStartupCriticalSection::critSec);
    status = GdiplusNotInitialized;
  }
  __asm { fclex }
  return status;
}

The code of this method is absolutely clear. Its essence lies in the lines:

if ( result && point1 && point2 && wrapMode != 4 )
{
  vColor1 = color1;
  vColor2 = color2;
  v8 = operatornew(a1);
  status = 0;
  if ( v8 )
    v9 = GpLineGradient::GpLineGradient(v8, point1, point2, &vColor1, &vColor2, wrapMode);
  else
    v9 = 0;
  *result = v9;
  if ( !CheckValid<GpHatch>(result) )
    status = OutOfMemory
} 
else {
  status = InvalidParameter;
}

GdiPlus checks if the input parameters are valid, and, if not, returns an InvalidParameter. Otherwise, GpLineGradient is created and checked for validity. If validation fails, OutOfMemory is returned. Apparently, this is our case, which means that we need to understand what is happening inside the GpLineGradient constructor:

GpLineGradient :: GpLineGradient

GpRectGradient *__thiscall GpLineGradient::GpLineGradient(GpGradientBrush *this, GpPointF *point1, GpPointF *point2, int color1, int color2, int wrapMode)
{
  GpGradientBrush *v6; // esifloat height; // ST2C_4double v8; // st7float width; // ST2C_4float angle; // ST2C_4
  GpRectF rect; // [esp+1Ch] [ebp-10h]
  v6 = this;
  GpGradientBrush::GpGradientBrush(this);
  GpRectGradient::DefaultBrush(v6);
  rect.Height = 0.0;
  rect.Width = 0.0;
  rect.Y = 0.0;
  rect.X = 0.0;
  *v6 = &GpLineGradient::`vftable;
  if ( LinearGradientRectFromPoints(point1, point2, &rect) )
  {
    *(v6 + 1) = 1279869254;
  }
  else
  {
    height = point2->Y - point1->Y;
    v8 = height;
    width = point2->X - point1->X;
    angle = atan2(v8, width) * 180.0 / 3.141592653589793;
    GpLineGradient::SetLineGradient(v6, point1, point2, &rect, color1, color2, angle, 0, wrapMode);
  }
  return v6;
}

Here the variables are initialized, which are then filled in LinearGradientRectFromPoints and SetLineGradient. I dare to assume that rect is a fill rectangle based on point1 and point2, to make sure of this, you can look at LinearGradientRectFromPoints:

LinearGradientRectFromPoints

GpStatus __fastcall LinearGradientRectFromPoints(GpPointF *p1, GpPointF *p2, GpRectF *result){
  double vP1X; // st7float vLeft; // ST1C_4 MAPDSTdouble vP1Y; // st7float vTop; // ST1C_4 MAPDSTfloat vWidth; // ST18_4 MAPDSTdouble vWidth3; // st7float vHeight; // ST18_4 MAPDSTfloat vP2X; // [esp+18h] [ebp-8h]float vP2Y; // [esp+1Ch] [ebp-4h]if ( IsClosePointF(p1, p2) )
    return InvalidParameter;
  vP2X = p2->X;
  vP1X = p1->X;
  if ( vP2X <= vP1X )
    vP1X = vP2X;
  vLeft = vP1X;
  result->X = vLeft;
  vP2Y = p2->Y;
  vP1Y = p1->Y;
  if ( vP2Y <= vP1Y )
    vP1Y = vP2Y;
  vTop = vP1Y;
  result->Y = vTop;
  vWidth = p1->X - p2->X;
  vWidth = fabs(vWidth);
  vWidth3 = vWidth;
  result->Width = vWidth;
  vHeight = p1->Y - p2->Y;
  vHeight = fabs(vHeight);
  result->Height = vHeight;
  vWidth = vWidth3;
  if ( IsCloseReal(p1->X, p2->X) )
  {
    result->X = vLeft - 0.5 * vHeight;
    result->Width = vHeight;
    vWidth = vHeight;
  }
  if ( IsCloseReal(p1->Y, p2->Y) )
  {
    result->Y = vTop - vWidth * 0.5;
    result->Height = vWidth;
  }
  return0;
}

As expected, rect is a rectangle of points1 and point2.

Now back to our main problem and see what happens inside SetLineGradient:

SetLineGradient

GpStatus __thiscall GpLineGradient::SetLineGradient(DpGradientBrush *this, GpPointF *p1, GpPointF *p2, GpRectF *rect, int color1, int color2, float angle, int zero, int wrapMode)
{
  _DWORD *v10; // edifloat *v11; // edi
  GpStatus v12; // esi
  _DWORD *v14; // edithis->wrapMode = wrapMode;
  v10 = &this->dword40;
  this->Color1 = *color1;
  this->Color2 = *color2;
  this->Color11 = *color1;
  this->Color21 = *color2;
  this->dwordB0 = 0;
  this->float98 = 1.0;
  this->dwordA4 = 1;
  this->dwordA0 = 1;
  this->float94 = 1.0;
  this->dwordAC = 0;
  if ( CalcLinearGradientXform(zero, rect, angle, &this->gap4[16]) )
  {
    *this->gap4 = 1279869254;
    *v10 = 0;
    v14 = v10 + 1;
    *v14 = 0;
    ++v14;
    *v14 = 0;
    v14[1] = 0;
    *&this[1].gap4[12] = 0;
    *&this[1].gap4[16] = 0;
    *&this[1].gap4[20] = 0;
    *&this[1].gap4[24] = 0;
    *&this->gap44[28] = 0;
    v12 = InvalidParameter;
  }
  else
  {
    *this->gap4 = 1970422321;
    *v10 = LODWORD(rect->X);
    v11 = (v10 + 1);
    *v11 = rect->Y;
    ++v11;
    *v11 = rect->Width;
    v11[1] = rect->Height;
    *&this->gap44[28] = zero;
    v12 = 0;
    *&this[1].gap4[12] = *p1;
    *&this[1].gap4[20] = *p2;
  }
  return v12;
}

In SetLineGradient, only fields are initialized. So, we need to go deeper:

int __fastcall CalcLinearGradientXform(int zero, GpRectF *rect, float angle, int a4){
  //...//...//...return GpMatrix::InferAffineMatrix(a4, points, rect) != OK ? InvalidParameter : OK;
}

And finally:

GpStatus __thiscall GpMatrix::InferAffineMatrix(intthis, GpPointF *points, GpRectF *rect)
{
  //...double height; // st6double y; // st5double width; // st4double x; // st3double bottom; // st2float right; // ST3C_4float rectArea; // ST3C_4//...
  x = rect->X;
  y = rect->Y;
  width = rect->Width;
  height = rect->Height;
  right = x + width;  
  bottom = height + y;
  rectArea = bottom * right - x * y - (y * width + x * height);
  rectArea = fabs(rectArea);
  if ( rectArea < 0.00000011920929 )
    return InvalidParameter;
  //...
}

In the InferAffineMatrix method, exactly what interests us is happening. Here the rect area is checked - the original rectangle of the points, and if it is less than 0.00000011920929, then the InferAffineMatrix returns an InvalidParameter. 0.00000011920929 is a machine epsilon for float (FLT_EPSILON). You can see how interesting Microsoft considers the area of a rectangle:

rectArea = bottom * right - x * y - (y * width + x * height);

From the area to the bottom right corner, subtract the area to the top left, then subtract the area above the rectangle and to the left of the rectangle. Why this is done, I do not understand; I hope someday I will come to know this secret method.

So, what we have:

InnerAffineMatrix returns an InvalidParameter;
CalcLinearGradientXForm throws this result higher;
In SetLineGradient, execution will follow the if branch, and the method will also return an InvalidParameter;
The GpLineGradient constructor will lose the information about the InvalidParameter and return the GpLineGradient object that is not initialized to the end - this is very bad!
GdipCreateLineBrush will check in CheckValid (line 26) the GpLineGradient object with the fields empty to the end and regularly return false.
After that, status will change to OutOfMemory, which will get .NET at the output of the GDI + method.

It turns out that Microsoft for some reason ignores the return status of some methods, makes because of this incorrect assumptions and complicates the understanding of the work of the library for other programmers. But after all, it was necessary to forward the status higher from the GpLineGradient constructor, and in GdipCreateLineBrush to check the return value on OK and otherwise return the constructor status. Then for GDI + users, an error message that occurred inside the library would look more logical.

The option of replacing very small numbers with zero, i.e. Vertically filled, runs without error due to the magic that Microsoft performs in the LinearGradientRectFromPoints method on lines 35 through 45:

Magic

if ( IsCloseReal(p1->X, p2->X) )
{
  result->X = vLeft - 0.5 * vHeight;
  result->Width = vHeight;
  vWidth = vHeight;
}
if ( IsCloseReal(p1->Y, p2->Y) )
{
  result->Y = vTop - vWidth * 0.5;
  result->Height = vWidth;
}

How to treat?

How to avoid this fall in .NET code? The simplest and most obvious option is to compare the area of the rectangle of points1 and point2 with FLT_EPSILON and not create a gradient if the area is smaller. But with this option, we will lose the information on the gradient, and an unfilled area will be drawn, which is not good. I see a more acceptable option, when the angle of the gradient fill is checked, and if it turns out that the fill is close to horizontal or vertical, then we set the same parameters for the points:

My C # solution

static LinearGradientBrush CreateBrushSafely(PointF p1, PointF p2) {
    if(IsShouldNormalizePoints(p1, p2)) {
        if(!NormalizePoints(ref p1, ref p2))
            returnnull;
    }
    var brush = new LinearGradientBrush(p1, p2, Color.White, Color.Black);
    return brush;
}
staticboolIsShouldNormalizePoints(PointF p1, PointF p2) {
    float width = Math.Abs(p1.X - p2.X);
    float height = Math.Abs(p1.Y - p2.Y);
    return width * height < FLT_EPSILON && !(IsCloseFloat(p1.X, p2.X) || IsCloseFloat(p1.Y, p2.Y));
}
staticboolIsCloseFloat(float v1, float v2) {
    var t = v2 == 0.0f ? 1.0f : v2;
    return Math.Abs((v1 - v2) / t) < FLT_EPSILON;
}
staticboolNormalizePoints(ref PointF p1, ref PointF p2) {
    constdouble twoDegrees = 0.03490658503988659153847381536977d;
    float width = Math.Abs(p1.X - p2.X);
    float height = Math.Abs(p1.Y - p2.Y);
    var angle = Math.Atan2(height, width);
    if (Math.Abs(angle) < twoDegrees) {
        p1.Y = p2.Y;
        returntrue;
    }
    if (Math.Abs(angle - Math.PI / 2) < twoDegrees) {
        p1.X = p2.X;
        returntrue;
    }
    returnfalse;
}

And how are the competitors?

Let's find out what happens in Wine. To do this, look at the source code of Wine , line 306:

Wine's GdipCreateLineBrush

/******************************************************************************
 * GdipCreateLineBrush [GDIPLUS.@]
 */GpStatus WINGDIPAPI GdipCreateLineBrush(GDIPCONST GpPointF* startpoint,
    GDIPCONST GpPointF* endpoint, ARGB startcolor, ARGB endcolor,
    GpWrapMode wrap, GpLineGradient **line){
    TRACE("(%s, %s, %x, %x, %d, %p)\n", debugstr_pointf(startpoint),
          debugstr_pointf(endpoint), startcolor, endcolor, wrap, line);
    if(!line || !startpoint || !endpoint || wrap == WrapModeClamp)
        return InvalidParameter;
    if (startpoint->X == endpoint->X && startpoint->Y == endpoint->Y)
        return OutOfMemory;
    *line = heap_alloc_zero(sizeof(GpLineGradient));
    if(!*line)  return OutOfMemory;
    (*line)->brush.bt = BrushTypeLinearGradient;
    (*line)->startpoint.X = startpoint->X;
    (*line)->startpoint.Y = startpoint->Y;
    (*line)->endpoint.X = endpoint->X;
    (*line)->endpoint.Y = endpoint->Y;
    (*line)->startcolor = startcolor;
    (*line)->endcolor = endcolor;
    (*line)->wrap = wrap;
    (*line)->gamma = FALSE;
    (*line)->rect.X = (startpoint->X < endpoint->X ? startpoint->X: endpoint->X);
    (*line)->rect.Y = (startpoint->Y < endpoint->Y ? startpoint->Y: endpoint->Y);
    (*line)->rect.Width  = fabs(startpoint->X - endpoint->X);
    (*line)->rect.Height = fabs(startpoint->Y - endpoint->Y);
    if ((*line)->rect.Width == 0)
    {
        (*line)->rect.X -= (*line)->rect.Height / 2.0f;
        (*line)->rect.Width = (*line)->rect.Height;
    }
    elseif ((*line)->rect.Height == 0)
    {
        (*line)->rect.Y -= (*line)->rect.Width / 2.0f;
        (*line)->rect.Height = (*line)->rect.Width;
    }
    (*line)->blendcount = 1;
    (*line)->blendfac = heap_alloc_zero(sizeof(REAL));
    (*line)->blendpos = heap_alloc_zero(sizeof(REAL));
    if (!(*line)->blendfac || !(*line)->blendpos)
    {
        heap_free((*line)->blendfac);
        heap_free((*line)->blendpos);
        heap_free(*line);
        *line = NULL;
        return OutOfMemory;
    }
    (*line)->blendfac[0] = 1.0f;
    (*line)->blendpos[0] = 1.0f;
    (*line)->pblendcolor = NULL;
    (*line)->pblendpos = NULL;
    (*line)->pblendcount = 0;
    linegradient_init_transform(*line);
    TRACE("<-- %p\n", *line);
    return Ok;
}

Here is the only validation parameter check:

if(!line || !startpoint || !endpoint || wrap == WrapModeClamp)
    return InvalidParameter;

Most likely, the following was written for compatibility with Windows:

if (startpoint->X == endpoint->X && startpoint->Y == endpoint->Y)
    return OutOfMemory;

And the rest is nothing interesting - the allocation of memory and filling the fields. From the source code, it becomes obvious that in Wine, the creation of a problematic gradient fill should be done without errors. And really - if you run the following program in Windows (I ran in Windows10x64)

Test program

#include<Windows.h>#include"stdafx.h"#include<gdiplus.h>#include<iostream>#pragma comment(lib,"gdiplus.lib")voidCreateBrush(float x1, float x2){
 Gdiplus::LinearGradientBrush linGrBrush(
  Gdiplus::PointF(x1, -0.5f),
  Gdiplus::PointF(x2, 10.5f),
  Gdiplus::Color(255, 0, 0, 0),
  Gdiplus::Color(255, 255, 255, 255));
 constint status = linGrBrush.GetLastStatus();
 constchar* result;
 if (status == 3) {
  result = "OutOfMemory";
 }
 else {
  result = "Ok";
 }
 std::cout << result << "\n";
}
intmain(){
 Gdiplus::GdiplusStartupInput gdiplusStartupInput;
 ULONG_PTR gdiplusToken;
 Gdiplus::GdiplusStartup(&gdiplusToken, &gdiplusStartupInput, NULL);
 Gdiplus::Graphics myGraphics(GetDC(0));
 CreateBrush(-3.367667E-16f, 3.367667E-16f);
 CreateBrush(0, 0);
    return0;
}

That in the Windows console will be:

OutOfMemory
Ok

and in Ubuntu with Wine:

Ok
ok

It turns out that either I am doing something wrong, or Wine in this matter is more logical than Windows.

Conclusion

I really hope that I did not understand something and the behavior of GDI + is logical. True, it is not at all clear why Microsoft did just that. I’ve been digging into their other products a lot, and there too there are things that, in a decent society, would not have passed the Code Review.

Tags: