Fighting memory leaks (C ++ CRT)

A memory leak is a pretty serious and dangerous problem. Perhaps the user will not notice a single leak of any 32Kb of memory (and this is the whole 5% of 640Kb, which "is enough for everyone" ), but constantly losing complex hierarchical structures or arrays larger than INT_MAX( which we like to create on 64- bit architecture ) we will doom it to suffering, and our product to failure.

It seems that it is not difficult to prevent the situation - we will use the “put everything in place” rule, but in practice this is greatly complicated by the human factor (banal carelessness), architecture cunning and non-linear order of operator execution, for example, due to the use of exceptions.

And you could just "surrender" automatic garbage collector, the cost of lost productivity (and this is not necessarily Managed C ++, for Native C ++ / C has a garbage collection library, here , for example).

But we will talk about the situation when “everything is bad” already.

Then the task is reduced to detecting and correcting possible leaks - as for the correction, everything is usually simple ( deleteor delete[]) here. But how to detect a leak? Google will happily give answers that usually come down to the fact that:

• need to use third-party leak analyzers
• have to reinvent the wheel and scooters
• it would be nice to write your own memory manager

But it is possible and easier, using Debug CRT.

Step 1. Enabling leakage accounting

To do this, connect the Debug CRT header and enable the use of the Debug Heap Alloc Map:

#ifdef _DEBUG
#include
#define _CRTDBG_MAP_ALLOC
#endif

* This source code was highlighted with Source Code Highlighter.

Now, when allocating memory through newor, the malloc()data is wrapped in the following structure ( but in fact I am a little cunning, the field responsible for data does not correspond to the syntax structand the “structure” itself is defined somewhere inside the CRT and its description is not available to the programmer ):

typedef struct _CrtMemBlockHeader
{
  struct _CrtMemBlockHeader * pBlockHeaderNext;
  struct _CrtMemBlockHeader * pBlockHeaderPrev;
  char* szFileName;
  int  nLine;
  size_t nDataSize;
  int nBlackUse;
  long lRequest;
  unsigned char gap[nNoMansLandSize];  
  unsigned char data[nDataSize];
  unsigned char anotherGap[nNoMansLandSize];
} _CrtMemBlockHeader;

* This source code was highlighted with Source Code Highlighter.

It contains information about the file name szFileNameand the line nLinewhere the memory was allocated, the amount of requested memory nDataSize, and, in fact, the data itself data, wrapped in the so-called No Mans Land area. Themselves are BlockHeaderorganized in a doubly linked list, which makes it easy to list them, and, accordingly, identify all memory allocation operations for which there was no corresponding release operation.

Step 2. Listing Leaks

We need a function that will go over the list of CrtMemBlockHeaders and give us information about the problem areas:

_CrtDumpMemoryLeaks();

Then in the Debug Output window we will see the following information:

Detected memory leaks!
Dumping objects ->
{163} normal block at 0x00128788, 4 bytes long.
Data: <  > 00 00 00 00
{162} normal block at 0x00128748, 4 bytes long.
Data: <  > 00 00 00 00
Object dump complete.

* This source code was highlighted with Source Code Highlighter.

And it's almost cool, but this result is not yet usable for several reasons:

• It does not contain information about the file and the line where the memory was allocated (and there is such information in the structure!)
• I would very much like to output it to some kind of log file (at least some automation)
• It contains redundant information, that is, not only memory that has already "leaked" ...

... and also the one that simply did not have time to "return" from global objects. And maybe global objects are bad, but now we get used to the idea that they are, which means _CrtDumpMemoryLeaks()we need to somehow remove them from the output . And this is solved by the following trick:

int _tmain(int argc, _TCHAR* argv[])
{
  _CrtMemState _ms;
  _CrtMemCheckpoint(&_ms);
 
  // some logic goes here...
  
   _CrtMemDumpAllObjectsSince(&_ms);
  return 0;
}

* This source code was highlighted with Source Code Highlighter.

We write in a special structure the initial (current at the time of entering main) memory state ( _CrtMemCheckpoint), and before the application terminates, we display all the remaining objects in memory ( _CrtMemDumpAllObjectsSince) created after the moment _ms- they are "leaks". Now the information is correct, we will take care of its convenience.

Step 3. Presentation of the results

Redirecting the output is very easy, the functions _CrtSetReportModeand help us here _CrtSetReportFile.

_CrtSetReportMode( _CRT_WARN, _CRTDBG_MODE_FILE );
_CrtSetReportFile( _CRT_WARN, _CRTDBG_FILE_STDOUT );

* This source code was highlighted with Source Code Highlighter.

Now the output of all warnings (which is any output _CrtMemDumpAllObjectsSince) will go straight to stdout. The second parameter of the function _CrtSetReportFilecan also be the real handle of the file.

Why aren't the file names and lines where memory allocation occurred? It so happened that, according to Microsoft Visual C ++ 6.0, the following function override newin the header was responsible for this information crtdbg.h:

#ifdef _CRTDBG_MAP_ALLOC
  inline void* __cdecl operator new(unsigned int s)
     { return ::operator new(s, _NORMAL_BLOCK, __FILE__, __LINE__); }
  #endif /* _CRTDBG_MAP_ALLOC */

* This source code was highlighted with Source Code Highlighter.

And, it’s easy to guess, this did not give the desired result, they were __FILE__:__LINE__always deployed in “crtdbg.h file line 512” . And then the guys from Microsoft generally removed this "feature", giving it to the programmer. Well, it's not scary, because you can achieve this functionality with one definition:

#define new new( _NORMAL_BLOCK, __FILE__, __LINE__)

* This source code was highlighted with Source Code Highlighter.

It is highly advisable to put it into some common header file (you must include it later crtdbg.h). Problems will arise if it newhas already been redefined. Although, as I see it, some reasonable redefinition of new will not use CRT (otherwise it would be possible to use the hook technique), and the scheme, in this case, will not be applicable at all, well, okay.

In general, now they got what they wanted: here is an example of a conclusion, but, I think, it is already obvious what should be there.

Reckoning hour

Of course, the organization and support of CRT Internals structures takes time and additional memory. How much is it?

UPD: Everything below is valid only for Win32 (tested on Vista SP1).

We create 10 million int using new(40Mb of memory theoretically):

Debug CRT	~ 500Mb	3 seconds
Release	~ 160Mb	1 second

The ~ 160Mb figure for the release build may surprise you a bit. But this is normal - it newallocates memory through the OS function HeapAlloc, which aligns data at multiple 16 addresses (for Win32). Allocating memory for one character, we get another 15 bytes, with which you can even do (but certainly do not need to do) something bad. For the debug, a very predictable result - we add another sizeof(_CrtMemBlockHeader)10 million times and get just 500 megabytes.

An interesting empirical result turned out that the release new intworks somewhere 10% slower than HeapAlloc4 bytes, is hardly distinguishable by speed from new int()(initialization by default, that is, zero), and 5-10% faster than HeapAllocwith a flag HEAP_ZERO_MEMORY.

Well, now 128 thousand int [256] throughnew int[256] (128Mb of memory theoretically):

Debug CRT	~ 136Mb	172 ms
Release	~ 128.5Mb	60 ms

The results are predictable and quite satisfactory. A speed ratio of 1: 3 was also confirmed on data of a different size, including when mixing various data and partially freeing memory. But without Debug dynamic memory operations, the code runs several times slower than Release!

Conclusion

Memory leaks can be dealt with with bare hands. Of course, our “raw” output will not be as effective as the leak tree, or a list of code places sorted in descending order of the total leak (although this can all be generated without difficulty according to our conclusion). But for small projects or tasks can do the trick. And the method doesn’t need support (it’s not quite “written and forgot” because of redefinition new, but close to that ), and the level of entry is much lower than for serious analyzers.

Well, perhaps that's all. Is that the source for the reconstruction of a holistic picture.

UPD: The method is not going to compete with external analyzers, as the goals are somewhat different, but mention of standing tools is very welcome (only, please, without repetition).