From translator

High-level languages ​​have won in the world and in the worlds of ruby-python-js developers it remains only to rant that it is not worth using this or that in the pros. For example, exceptions, because they are slow and generate a lot of redundant code. It was worth asking "and what code does it generate," when in response it received a mumble and a lowing. And the truth is - how do they work? Well, we compile in g ++ with the -S flag, we look at what happened. It’s not difficult to understand superficially, but the fact that there were misunderstandings did not let me sleep. Fortunately, a finished article was found.

There are several articles on the Habré, detailed and not very(but still good ones) dedicated to how exceptions work in C ++. However, there is not one truly deep, so I decided to fill this gap, since there is suitable material. Who cares how the exceptions in C ++ work on the example of gcc - stock up on pocket or evernote, free time and welcome to cat.

2 part
3 part

PS A few words about the translation:

• the translation is very very close to the text, but sometimes I allowed myself to change whole paragraphs
• I didn’t think of some terms how to translate, for example landing pad and call site
• The work turned out to be much more than it seemed, by the end I even began to confuse where the translation is, and where is the original, some lines were written at 4 nights, in general - if there are somewhere incoherent words or whole sentences - sorry, I’ll try in the near future fix it all.
• In this case, the code is an integral part of the article, so I will not hide anything under the spoiler.
• As always, spelling, punctuation and minor mistakes - in PM. Actual errors, inaccuracies and omissions are in the comment.

C ++ exceptions under the hood

Everyone knows that handling exceptions is difficult. There are plenty of reasons for this in each layer of the “life cycle” of an exception: it is difficult to write code with a strong exception safe code, exceptions can be thrown from unexpected places, trying to understand a poorly designed exception hierarchy can be problematic, it slowly works because for a large amount of voodoo magic under the hood, this is dangerous, as improperly throwing an error can lead to an unforgivable challenge std::terminate. And despite all this, the battle over whether or not to use exceptions in programs is still ongoing. This is probably due to a shallow understanding of how they work.

First you need to ask yourself: how does it all work? This is the first article in a long series.which I write about how exceptions are implemented under the hood in C ++ (under the gcc platform under x86, but should be applicable for other platforms as well). In these articles, the process of throwing and catching errors will be explained in detail, but for the impatient: a short brief of all articles on throwing exceptions in gcc / x86:

1. When we write the throw statement, the compiler translates it into a couple of function calls libstdc++that throw an exception and begin the fast process of unwinding the stack with a library call libstdc.
   
   
2. For each catch block, the compiler appends some special information after the method body, an exception table that the method can catch, and a cleanup table (more on the cleanup table below).
   
   
3. In the process of unwinding the stack, a special function is libstdc++called , which is supplied (called the "personality routine"), which checks each function on the stack for errors that it can catch.
   
   
4. If there is no one who could catch this error, it is called std::terminate.
   
   
5. If someone is still found, the promotion starts again from the top of the stack.
   
   
6. Upon repeated passage through the stack, a “personal function” is started to clear resources for each method.
   
   
7. The routine checks the cleanup table for the current method. If it has something to clear, the routine “jumps” to the current frame of the stack and launches a cleanup code that calls destructors for each of the objects located in the current scope.
   
   
8. When the promotion comes across a fragment of the stack that can handle the exception, it jumps into the exception processing block.
   
   
9. After the exception is processed, the cleanup function is called to free the memory occupied by the exception.

* This will be one big article for us, broken into pieces, therefore the “series of articles” will be replaced simply with the “article” in order not to clutter up the clutter.

Even now, it looks complicated, but we didn’t even start, it was only a short and inaccurate description of the difficulties required to handle exceptions.

To study all the details happening under the hood, in the next part we will start by implementing our own mini-version libstdlibc++. Not all, only parts with error handling. In reality, not even all of this part is just a necessary minimum for implementing a throw / catch block. You will also need a little assembler, but only quite a bit. But it will take a lot of patience, unfortunately.

If you are too curious, you can start here.. This is a complete specification of what we will implement in the following parts. I’ll try to make this article instructive and simpler so that next time it’s easier for you to start with your own ABI (application binary interface).

Notes (disclaimer):
I will in no way know what kind of voodoo magic happens when an exception is thrown. In this article I will try to expose the secret and find out how it works. Some little things and subtleties will not correspond to reality. Please let me know if something is wrong somewhere.

Note translator: this is also true for translation.

C ++ exceptions under the hood: small ABI

If we try to understand why exceptions are so complex and how they work, we can either drown in tons of manuals and documentation, or try to catch the exceptions ourselves. In fact, I was surprised at the lack of quality information on the topic (translator's note - I, by the way, too): everything that can be found is either too detailed, or too simple. Of course, there are specifications (the most documented: ABI for C ++ , but also CFI , DWARF and libstdc), but a separate reading of the documentation is not enough if you really want to understand what is happening inside.

Let's start with the obvious: reinventing the wheel! We know that in pure C there are no exceptions, so let's try to link the C ++ program with the linker of pure C and see what happens! I started with something simple like this:

#include "throw.h"
extern "C" {
    void seppuku() {
        throw Exception();
    }
}

Don’t forget extern, otherwise G ++ will helpfully cut out our small function and make it impossible to link with our program on pure C. Of course, we need a header file for linking (not a pun) to enable the connection of the worlds of C ++ and C:

struct Exception {};
#ifdef __cplusplus
extern "C" {
#endif
    void seppuku();
#ifdef __cplusplus
}
#endif

And a very simple main:

#include "throw.h"
int main()
{
    seppuku();
    return 0;
}

What happens if we try to compile and link this frankcode?

> g++ -c -o throw.o -O0 -ggdb throw.cpp
> gcc -c -o main.o -O0 -ggdb main.c

Note: you can download all the source code for this project from my git repository .

So far, so good. Both g ++ and gcc are happy in their small world. Chaos will begin as soon as we try to link them together:

> gcc main.o throw.o -o app
throw.o: In function `foo()':
throw.cpp:4: undefined reference to `__cxa_allocate_exception'
throw.cpp:4: undefined reference to `__cxa_throw'
throw.o:(.rodata._ZTI9Exception[typeinfo for Exception]+0x0): undefined reference to `vtable for __cxxabiv1::__class_type_info'
collect2: ld returned 1 exit status

And of course, gcc complains about missing C ++ declarations. These are very specific C ++ declarations. Look at the last line of the error: skipped vtablefor cxxabiv1. cxxabideclared in libstdc++refers to the ABI for C ++. Now we know that error handling is performed using the standard C ++ library with the declared C ++ ABI interface.

C ++ ABI announces a standard binary format with which we can link objects together in one program. If we compile .o files with two different compilers that use different ABIs, we cannot combine them into one application. The ABI may also declare various other standards, for example, an interface for unwinding a stack or throwing an exception. In this case, the ABI defines an interface (not necessarily a binary format, just an interface) between C ++ and other libraries in our application that provide stack promotion. In other words, ABI defines C ++-specific things thanks to which our application can communicate with non-C ++ libraries: this is what will allow us to throw exceptions from other languages ​​that will be caught in C ++, and many other things.

In any case, linker errors are the point of departure and the first layer in the analysis of the operation of exceptions under the hood: the interface we need to implement is cxxabi. In the next chapter, we will start with our own mini-ABI, defined exactly as C ++ ABI .

C ++ exceptions under the hood: please the linker by pushing the ABI

On our journey in understanding exceptions, we discovered that all weightlifting is implemented in libstdc++, the definition of which is given in C ++ ABI. Looking through the errors of the linker, we deduced that for error handling we should turn to C ++ ABI for help; we created a C ++ program spitting errors, linked with a pure C program and found that the compiler somehow translates our throw instructions into something that now calls several libstd ++ functions that directly throw an exception.

Nevertheless, we want to understand exactly how exceptions work, so let's try to implement our own mini-ABI, which provides an error throwing mechanism. To do this, we only need RTFM , however the full interface can be found here for LLVM. Recall exactly what features are missing:

> gcc main.o throw.o -o app
throw.o: In function `foo()':
throw.cpp:4: undefined reference to `__cxa_allocate_exception'
throw.cpp:4: undefined reference to `__cxa_throw'
throw.o:(.rodata._ZTI9Exception[typeinfo for Exception]+0x0): undefined reference to `vtable for __cxxabiv1::__class_type_info'
collect2: ld returned 1 exit status

__cxa_allocate_exception

The name is self-sufficient, I suppose. __cxa_allocate_exception acceptssize_tand allocates enough memory to hold the exception while throwing it. This is more complicated than it seems: when the error is processed, there is some magic with the stack, allocation (comment of the translator - forgive this word, but sometimes I will use it) on the stack is a bad idea. Allocating memory on the heap (heap), in general, is also a bad idea, because where will we allocate memory with an exception, signaling that the memory has run out? Static storage in memory is also a bad idea, as long as we need to make it thread safe (otherwise two competing threads that throw exceptions will lead to disaster). Given these problems, the allocation of memory in the local stream storage (heap) looks most advantageous, however, if necessary, access the emergency storage (presumably static), if the memory is out of memory. Of course, we won’t worry about the scary details, so we can just use a static buffer if necessary.

__cxa_throw

This feature does all the magic of forwarding! According to ABI, once an exception has been raised, __cxa_throw should be called . This function is responsible for invoking the stack promotion. Important effect: __cxa_throw never implies a return. It also transfers control to the appropriate catch block to handle the exception or calls (by default) std::terminate, but never returns anything.

vtable for __cxxabiv1::__class_type_info

Strange ... __class_type_info is clearly some RTTI (run-time type information, run-time type identification, Dynamic data type identification), but which one? So far, it’s not easy for us to answer this, and it’s not hellishly important for our mini-ABI; let’s leave this part of the “application”, which we will give after the analysis of the process of throwing an exception, now let's just say that this is the entry point of the ABI definition in runtime that answers the question: “are these two types the same or not”. This is a function that is called to determine whether a given catch block can handle this error or not. Now we will focus on the main thing: we need to give it as the address for the linker (i.e., defining it is not enough, we also need to initiate it) and it should have a vtable (yes yes,

A lot of work happens in these functions, but let's try to implement a simple exception thrower: one that will exit the program (call exit) when an exception is thrown. Our application is almost complete, but some ABI functions are missing, so let's create mycppabi.cpp. By reading our ABI specification , we can describe our signatures for __cxa_allocate_exception and __cxa_throw :

#include 
#include 
#include 
namespace __cxxabiv1 {
    struct __class_type_info {
        virtual void foo() {}
    } ti;
}
#define EXCEPTION_BUFF_SIZE 255
char exception_buff[EXCEPTION_BUFF_SIZE];
extern "C" {
void* __cxa_allocate_exception(size_t thrown_size)
{
    printf("alloc ex %i\n", thrown_size);
    if (thrown_size > EXCEPTION_BUFF_SIZE) printf("Exception too big");
    return &exception_buff;
}
void __cxa_free_exception(void *thrown_exception);
#include 
void __cxa_throw(
          void* thrown_exception,
          struct type_info *tinfo,
          void (*dest)(void*))
{
    printf("throw\n");
    // __cxa_throw never returns
    exit(0);
}
} // extern "C"

Let me remind you: you can find the sources in my github repository .

If we compile mycppabi.cpp now and link to the other two .o files, we get working binaries that should output "alloc ex 1 \ n throw" and, after that, exit. Very simple, but at the same time, surprising: we manage exceptions without calling libc ++: we wrote the (very very small) part of C ++ ABI!

Another important part of the wisdom we got when creating our own mini-ABI: the keyword throwcompiles into two function calls from libstdc ++. There is no voodoo magic; it is a simple transformation. We can even disassemble our function to test this. Rung++ -S throw.cpp

seppuku:
.LFB3:
    [...]
    call    __cxa_allocate_exception
    movl    $0, 8(%esp)
    movl    $_ZTI9Exception, 4(%esp)
    movl    %eax, (%esp)
    call    __cxa_throw
    [...]

Even more magic: when throwtranslated into these two calls, the compiler does not even know how the exception will be handled. As soon as it libstdc++determines __cxa_throwits friends, it is libstdc++dynamically linked in runtime, an exception handling method can be selected when the application is first launched.

We are already seeing progress, but we still have to go a long way of learning. Now our ABI can only throw exceptions. Can we expand it to catch errors? Well, let's see how to do this in the next chapter!

C ++ exceptions under the hood: catching what we throw

In this article, we slightly opened the veil of secrecy about throwing exceptions by observing errors of the compiler and linker, but we are still far from understanding anything about catching errors. We summarize what we have already found out:

• The throw-announcement will be broadcast by the compiler in two calls: __cxa_allocate_exception and __cxa_throw .
• __cxa_allocate_exception and __cxa_throw "live" in libstdc++.
• __cxa_allocate_exception allocates memory for a new exception.
• __cxa_throw prepares and throws an exception to _Unwind , into a set of functions that live in libstdcand performs the actual expansion of the stack ( ABI defines the interface of these functions).

Until now, it has been quite simple, but catching exceptions is a little more difficult, especially because it requires a bit of reflection (reflexion) (it allows the program to analyze its own code). Let's use our old method and add some catch block to our code, compile and see what happens:

#include "throw.h"
#include 
// Добавляем второй тип исключений
struct Fake_Exception {};
void raise() {
    throw Exception();
}
// Анализируем, что произойдет, если исключение не отлавливается в catch-блоке
void try_but_dont_catch() {
    try {
        raise();
    } catch(Fake_Exception&) {
        printf("Running try_but_dont_catch::catch(Fake_Exception)\n");
    }
    printf("try_but_dont_catch handled an exception and resumed execution");
}
// И что произойдет, если отлавилвается
void catchit() {
    try {
        try_but_dont_catch();
    } catch(Exception&) {
        printf("Running try_but_dont_catch::catch(Exception)\n");
    } catch(Fake_Exception&) {
        printf("Running try_but_dont_catch::catch(Fake_Exception)\n");
    }
    printf("catchit handled an exception and resumed execution");
}
extern "C" {
    void seppuku() {
        catchit();
    }
}

As before, we have a seppuku function connecting C and C ++ worlds, only this time we added several function calls to make our stack more interesting, we also added try / catch branches of blocks, so now we can analyze how libstdc ++ processes them.

And again we get linker errors about missing ABI functions:

> g++ -c -o throw.o -O0 -ggdb throw.cpp
> gcc main.o throw.o mycppabi.o -O0 -ggdb -o app
throw.o: In function `try_but_dont_catch()':
throw.cpp:12: undefined reference to `__cxa_begin_catch'
throw.cpp:12: undefined reference to `__cxa_end_catch'
throw.o: In function `catchit()':
throw.cpp:20: undefined reference to `__cxa_begin_catch'
throw.cpp:20: undefined reference to `__cxa_end_catch'
throw.o:(.eh_frame+0x47): undefined reference to `__gxx_personality_v0'
collect2: ld returned 1 exit status

We again see a bunch of interesting things. Call __cxa_begin_catch and __cxa_end_catch we expected, though for now and do not know what they are, but we can assume that they are equivalent to throw statement / __ cxa_allocate / throw statement . __gxx_personality_v0 is something new, and it will be the main theme of the next parts.

What does a personal function do? (with translator - did not come up with a better name, tell me in the comments if you have any ideas). We already said something about her in the introduction, but next time we look at her in more detail, as well as at our two new friends: __cxa_begin_catch and __cxa_end_catch .

C ++ exceptions under the hood: the magic around __cxa_begin_catch and __cxa_end_catch

After examining how exceptions are thrown, we find ourselves on the path to examining how they are caught. In the previous chapter, we added a try-catch-block to our example application to see what the compiler does, and also got linker errors just like the last time we looked at what would happen if we add a throw-block. Here is what the linker writes:

> g++ -c -o throw.o -O0 -ggdb throw.cpp
> gcc main.o throw.o mycppabi.o -O0 -ggdb -o app
throw.o: In function `try_but_dont_catch()':
throw.cpp:12: undefined reference to `__cxa_begin_catch'
throw.cpp:12: undefined reference to `__cxa_end_catch'
throw.o: In function `catchit()':
throw.cpp:20: undefined reference to `__cxa_begin_catch'
throw.cpp:20: undefined reference to `__cxa_end_catch'
throw.o:(.eh_frame+0x47): undefined reference to `__gxx_personality_v0'
collect2: ld returned 1 exit status

Let me remind you that you can get the code on my git repository .

In theory (in our theory, of course), the catch block is translated into the __cxa_begin_catch / end_catch pair from libstdc ++, but also into something new, called a personal function , which we still don't know anything about.

Let's test our theory about __cxa_begin_catch and __cxa_end_catch . Compile throw.cpp with the -S flag and analyze the assembler code. There is a lot of interesting things, we will cut to the most necessary:

_Z5raisev:
    call    __cxa_allocate_exception
    call    __cxa_throw

Everything is going fine: we got the same definition for raise (), just throw an exception:

_Z18try_but_dont_catchv:
    .cfi_startproc
    .cfi_personality 0,__gxx_personality_v0
    .cfi_lsda 0,.LLSDA1

The definition for try_but_dont_catch () is cropped by the compiler. This is something new: a link to __gxx_personality_v0 and something else called LSDA . This seems like a minor definition, but in reality it is very important:

• the linker uses this for the CFI (call frame information) specification; CFI stores call frame information, here is its full specification. It is mainly used to spin the stack.
• LDSA (language specific data area) is a special area for each language used by a personal function to know what exceptions can be handled by this function.

We will talk about CFI and LSDA in the next chapter, do not forget about them, but now let's move on.

[...]
call    _Z5raisev
jmp .L8

Another elementalism: just call raiseand then jump to L8; L8 makes a normal return from a function. If it is raiseexecuted incorrectly, then the execution (somehow, we still do not know how!) Should not continue on the next instruction, but should proceed to the exception handler (which is called in ABI terms landing pads, more about this later).

   cmpl    $1, %edx
    je  .L5
.LEHB1:
    call    _Unwind_Resume
.LEHE1:
.L5:
    call    __cxa_begin_catch
    call    __cxa_end_catch

At first glance, this piece is a little complicated, but in reality everything is simple. The greatest amount of magic happens here: first, we check whether we can handle this exception, if not, we call it _Unwind_Resume, if we can, we call __cxa_begin_catchand __cxa_end_catch, after that, the function should continue normally and, thus, L8 will be executed (L8 right under our catch block ):

.L8:
    leave
    .cfi_restore 5
    .cfi_def_cfa 4, 4
    ret
    .cfi_endproc

Just a normal function return ... with some CFI garbage in it.

This is all for error handling, however, we still don't know how __cxa_begin / end_catch works ; we have ideas on how this pair forms what the landing pad calls - the place in the function where the exception handlers are located. What we do not know yet is how landing pads are searched. Unwind should somehow go through all the calls on the stack, check: does any call (stack frame for accuracy) have a valid block with landing pad that can handle this exception, and continue to execute on it.

This is an important achievement, and how it works we will find out in the next chapter.

C ++ exceptions under the hood: gcc_except_table and personal function

Earlier, we found out that throw translates to __cxa_allocate_exception / throw , and the catch block translates to __cxa_begin / end_catch , as well as something called CFI (call frame information) to search for landing pads - entry points for error handlers.

What we don't know so far is how _Unwind finds out where these landing pads are. When an exception is thrown through a bunch of functions in the stack, all CFIs allow the stack expansion program to find out what function is currently executing, and it is also necessary to find out which of the landing pads of the function allows us to handle this exception (and, by the way, we ignore functions with multiple try / catch blocks!).

To find out where this landing pads is located, something called gcc_except_table is used . This table can be found (with CFI garbage) after the end of the function:

.LFE1:
    .globl  __gxx_personality_v0
    .section    .gcc_except_table,"a",@progbits
    [...]
.LLSDACSE1:
    .long   _ZTI14Fake_Exception

This section .gcc_except_table - where all the information for detecting landing pads is stored, we will talk about this later when we analyze the personal function. For now, we just say what LSDA means - a zone with language-specific data, which a personal function checks for landing pads for a function (it is also used to launch destructors in the process of expanding the stack).

To summarize: for each function where there is at least one catch block, the compiler translates it into a couple of calls to cxa_begin_catch / cxa_end_catch and then the personal function called by __cxa_throw reads gcc_except_tablefor each method in the stack to search for something called LSDA. The personal function then checks to see if there is a block in LSDA that handles this exception, and also if there is some kind of clearing code (which runs destructors when necessary).

We can also make an interesting conclusion: if we use nothrow (or an empty throw statement), the compiler can omit gcc_except_tablefor the method. This way of implementing exceptions in gcc, which does not greatly affect performance, actually greatly affects the size of the code. What about catch blocks? If an exception is thrown when the nothrow specifier is declared, LSDA is not generated and the personal function does not know what to do. When a personal function does not know what to do, it calls the default error handler, which, in most cases, means that throwing an error from the nothrow method will end with std :: terminate.

Now that we have ideas on what a personal function does, can we implement it? Well, let's see!

Continuation