Using an alternative memory allocator in a C / C ++ project

This article is written primarily for C / C ++ programmers who use Visual Studio 2013 in their work. Since I, as they say, totally windows guy, I can not evaluate the usefulness of this article for programmers who do not use this environment in their work. So.

It is no secret that the standard allocator new / delete / malloc / free in the C / C ++ language does not shine with speed. Of course, it all depends on the implementation, but if we talk about it from Microsoft, then this is a fact. In addition, the standard implementation of the allocator has another fatal flaw - memory fragmentation. If your program is often allocated / deallocated, you may find that after a few hours your program crashed due to lack of memory, although there is still enough free memory - just as a result of fragmentation in the allocator pool there is not enough free space left. (This, incidentally, is an absolutely real case that occurred at one of the projects in which I was directly involved.)

Fortunately, there are allocators that lack both of these shortcomings. I tried dlmalloc at one time and since then I always use it in my projects.

I want to share with you a way to connect dlmalloc to a project in Visual Studio C / C ++.
The method that I use is remarkable in that it allows you to use an alternative allocator for absolutely all allocations that can only happen in your program. Yes, a simple way (i.e. replacing malloc calls with dlmalloc) does not achieve this effect. For example, you connected a third-party library that will allocate memory using malloc. Moreover, some calls to standard functions from stdlib also allocate memory with the malloc function and you have no way to prevent this ... Or is it? There is.

The essence of the method

The essence of the method is to force the linker to use your implementation of malloc / free / new / delete instead of the standard one. But how to do that? When I just started researching this issue, my first attempt was a rather stupid idea: to patch the body malloc / free in runtime in memory by placing the unconditional jmp on my code there. Is there any need to explain why this idea is stupid? Although everything worked, but this method did not bring joy. As a result, I came to another solution, namely, to forbid the linker to use the standard libcmt library at all, in which the standard allocator is located. But this method also had a significant drawback, namely, in this library there are a lot of other useful and not very functions, to write stubs to which was categorically impossible.
Then I began to explore the possibility of taking the standard library (literally the libcmt.lib file) and throwing out all the extra ones from it. It turned out that this is possible and in the end I use this method.

Technical details

First, execute the command:

lib.exe /LIST libcmt.lib

The output will be a list of obj files that this library contains. For libcmt.lib from Visual Studio 2013, this list looks something like this:

f:\binaries\Intermediate\vctools\crt_bld\SELF_X86\crt\prebuild\INTEL\mt_lib\chandler4.obj
f:\binaries\Intermediate\vctools\crt_bld\SELF_X86\crt\prebuild\INTEL\mt_lib\chandler4gs.obj
f:\binaries\Intermediate\vctools\crt_bld\SELF_X86\crt\prebuild\INTEL\mt_lib\chkesp.obj
f:\binaries\Intermediate\vctools\crt_bld\SELF_X86\crt\prebuild\INTEL\mt_lib\eh3valid.obj
f:\binaries\Intermediate\vctools\crt_bld\SELF_X86\crt\prebuild\INTEL\mt_lib\exsup.obj
f:\binaries\Intermediate\vctools\crt_bld\SELF_X86\crt\prebuild\INTEL\mt_lib\exsup2.obj
... (и так далее)

Fortunately, almost all functions for working with memory are in separate obj files, which, in fact, makes this method possible.
Those. we need to cut out all unnecessary obj files from the library body.
The lib.exe utility with the / remove switch does just what we need.

Implementation

Actually, I posted the source codes on the github .

If you already have Visual Studio 2013 installed, just run make_libcmt_nomem.cmd , which will do all the work and create a cropped libcmt_nomem.lib file that you can connect instead of the full-weight libcmt.

The script uses the unix grep utility in its work. If you do not have UnixUtils installed, I highly recommend doing this (for example, from here ).

But that is not all. We got rid of the standard allocator. But the trouble is that at the same time we got rid of some standard functionality, which, alas, is inseparable from the allocator. Therefore, I wrote the necessary stubs, which you can find in the include / crtfunc.h file (ibid., On the github)

Mode of application

1. We get the cropped version of the standard library using the make_libcmt_nomem.cmd script and put it in an accessible place for the linker;
2. We disable the use of the standard libcmt library in the project (Ignore Specific Default Libraries "" libcmt "in the options of the linker Configuration Properties-> Linker-> Input);
3. In any c ++ file in the project, do "#include crtfunc.h" from the sources;
4. We connect dlmalloc to the project.

I don’t write down each item in detail, because if you read this article and understand it, you don’t need details. The only moment: you should include crtfunc.h in the C ++ (not C) file. If your project is written in C, you should add an empty .cpp file to the project and include crtfunc.h in it. However, no one forbids you to pick up a file.

PS. In fact, not dlmalloc'om single. There are other, very worthy allocators. The source files are designed for dlmalloc, but this is not important. Minimal intervention in crtfunc.h can be achieved using any other allocator.

After the release of Visual Studio 2015, all of the above has lost relevance