Embedding in the Linux kernel: intercepting system calls
The term "system call" in programming and computer technology refers to the application turning to the kernel of the operating system (OS) to perform an operation. Due to the fact that such interaction is the main one, interception of system calls seems to be the most important stage of integration, as allows you to control a key component of the OS kernel - the system call interface, which, in turn, makes it possible to inspect application software requests for kernel services.
This article is a continuation of the previously announced cycle devoted to particular issues of the implementation of imposed security features, and, in particular, integration into software systems.
There are various ways to intercept system calls to the Linux kernel. First of all, it is worth noting that to intercept single system calls, the previously considered method of intercepting kernel functions can be used . Indeed, since most system calls are represented by the corresponding functions (for example, sys_open ), the task of intercepting them is equivalent to the task of intercepting these functions. However, with the increase in the number of intercepted system calls and the complexity of the "business logic", this approach may be limited.
A more universal way is to modify entries in the system call tables (more on the tables below), which contain pointers to functions that implement the logic of a system call. Tables are used by the kernel for scheduling, when a pointer to a handler function is selected from the corresponding table by the number of the system call requested by the application with its subsequent execution. Replacing such a pointer will allow you to change the logic of the kernel in terms of handling system calls. Looking ahead, it is worth noting that for the successful implementation of this method, the tables themselves will need to be somehow found, because they are not exported. Ultimately, intercepting a system call will consist of simply overriding the table element.
The most universal way of intercepting system calls was and remains to be the modification of the code of the system call manager so that pre- and post-processing of the context of the thread requesting any system service is provided. This option gives greater flexibility in comparison with the previous ones, as Introduces single points of state monitoring before and after the handler function.
Next, an example will be discussed in detail how to embed Linux kernel system calls into the interface by modifying the dispatcher code.
Dispatching system calls is a rather complicated process with many nuances, however, many details will be omitted in this article, because, with the exception of the dispatching process itself (fetching and executing the function corresponding to the system call), you don’t need to know anything else to implement .
Traditionally, the Linux kernel supports the following system call capabilities for the x86 architecture:
The following is an excellent illustration of a system call I borrowed, depending on the option used:
As you can see, 32-bit applications make system calls using the INT 80h and SYSENTER mechanisms, while 64-bit applications use SYSCALL. At the same time, there is support for the ability to execute 32-bit code in a 64-bit environment (the so-called compatibility mode - emulation / compatibility mode; kernel option
In the general case, when all possible mechanisms are represented in a 64-bit kernel, there are 4 entry points that determine what the logic of the corresponding dispatcher will be:
One way or another, when the application makes a system call, the kernel gains control. The system call manager for each of the considered cases has differences from the others, however, without loss of generality, their general structure can be considered using the system_call example :
As you can see, the first instruction (
The first line checks whether the number of the requested system call (register
Thus, the key point in the process of processing system calls is the dispatch command, which is a function call (
As noted, a universal way to integrate into the dispatcher will be to modify its code in such a way as to provide the ability to control the context of the thread before it performs the function of implementing the logic of the system call (processing), as well as after its execution (post-processing).
In order to implement the embedding in the described way, it is proposed to patch the dispatcher slightly by modifying the dispatch command (
Here
The following is the code for the service_syscall64 handler , used as an example to intercept
Как видно, он имеет рассматриваемую выше структуру. Точные значения указателей и смещений настраиваются в процессе загрузки модуля (об этом будет рассказано далее). Кроме того, в приведённом фрагменте присутствуют дополнительные элементы (
Осуществление встраивания в механизмы диспетчеризации системных вызовов ядра ОС Linux так или иначе предполагает необходимость решения следующих практических задач:
Determining the addresses of system call dispatchers
The presence of several dispatchers in the system implies the need to determine their addresses. It was noted above that each dispatcher corresponds to its own “method” of making a request for a system call. Therefore, the appropriate mechanisms will be used to determine the required addresses:
Thus, each of the required addresses is easily determined.
Determination addresses dispatch tables syscalls
As noted above, tables
For these purposes, it is rational to use a disassembler ( udis86) By sequentially sorting through the instructions, starting with the very first one, you can reach the desired command, the argument of which will be the address of the corresponding table. Due to the fact that the dispatcher structure is well established, it is possible to unambiguously determine the characteristics of the desired command (CALL with a length of 7 bytes) and with a high degree of reliability obtain the required table address value from it.
If for some reason this is not enough, you can strengthen the verification of the received address. To do this, for example, you can check whether the value in the cell with the number of the
Modification of dispatchers code
When modifying the code of system call dispatchers, you need to consider that their code is read-only (ReadOnly). In addition, code modification on a working system should be atomic, i.e. so that during the modification process there are no undefined states when any of the threads sees a partially completed record.
In a previous article, the correct way to write to a write-protected page using the creation of temporary mappings is discussed. There is no need to repeat something here. With regard to atomicity, this issue was also discussed earlier when the topic of interception of the functions of the nucleus was considered.
Thus, it is advisable to modify the write-protected code using temporary mappings , as well as a special interface of the Linux kernel -
Configuring handlers
In accordance with the presented embedding method, the code of each of the dispatchers is modified in such a way that the 7-byte dispatch command
In accordance with the structure of the handlers, they contain commands (JMP and CALL) that require specifying exact arguments (for example, return addresses or addresses of the system call table). Due to the fact that such values are not available at the stage of compilation or loading of the module, they must be affixed “manually”, after loading, before starting work.
Another important feature when setting up handlers is the need to ensure that the module can be unloaded while maintaining the system’s health. For these purposes, the handler code must remain in the system even after unloading the main module (more on this later).
Unload module
Unloading the module should be carried out while maintaining the health of the system. This means that, after the module has been unloaded, the system should function normally. This task is not trivial due to the fact that with the unloading of the module, all the code used in it is unloaded.
For example, you can imagine a situation where a thread making a system call has fallen asleep in the kernel. Until the moment when he wakes up, someone tries to unload the module. In principle, nothing prevents the system from doing this. As a result, when the thread in question wakes up and completes the requested system call, control will return to the appropriate handler (which is why it should not be unloaded).
However, not unloading the code of the handlers is not the only condition for maintaining the system's operability after unloading the module. It is worth recalling that the real system call in the handler was “wrapped” in a couple of calls to the ServiceTraceEnter and ServiceTraceLeave trace functions , the code of which was located in the unloaded module.
Therefore, in order not to get into a situation where, upon returning from a system call, the thread would try to call a function that is physically no longer there, it is necessary to re-modify the code of each handler, eliminating more invalid calls from it (in other words, clogging them with NOPs).
The following describes the structure of the kernel module that implements the Linux kernel kernel system call dispatch mechanism.
The key structure of the module is struct scentry - a structure containing the information necessary for embedding in the appropriate dispatcher. The structure contains the following fields:
Structures are combined into an array that determines how and with what parameters to embed:
With the exception of the indicated fields, the remaining elements of the structure are filled in automatically - the function is responsible for this
As you can see, first of all, an attempt is made to resolve the symbol name to its corresponding address (
Next, for the dispatcher found, the dispatch command (
The completion of the preparation phase is the creation of the handler code used by the dispatcher after its modification. Below is the stub_fixup function that does this:
As you can see, the main role of this function is to create a copy of the handlers and then configure them to current addresses. Disassembler is also actively used here. The simple structure of the handlers allows you to do without any complex logic. A signal to exit the loop is the detection of a command
The preparation phase is followed by the phase of implanting the code into the kernel code. This phase is quite simple and consists of writing one single instruction (
When the module is unloaded, the restore phase is first executed , which consists in restoring the code of the system call dispatchers, as well as modifying the code of the handlers:
As you can see, in the handler code, all found 5-byte CALL commands will be replaced by a sequence of NOPs, which will exclude attempts to execute non-existent code upon returning from a system call. This was discussed earlier.
The functions corresponding to the implant and restore phases are performed in the context
The final phase during unloading is the clenup phase , where internal resources (
It is worth noting once again that after the module is unloaded , the area containing the code for the handlers will always remain in the kernel memory . As noted earlier, this is a prerequisite for the correct operation of the system after unloading the module.
Thus, the basic principles of embedding the kernel system calls into the mechanism of system calls are analyzed using an example, and the possibility of intercepting them is illustrated.
For testing purposes, we intercept a system call
Assembly and loading of the module is carried out by standard means:
As a result
Correct development of the interception
Moreover, it is worth noting that the interception operation for 32-bit applications (for example, Skype) is also carried out correctly, which is confirmed by the presence of messages starting with I386, and not with X86_64. Thus, the example
The method of embedding the Linux OS kernel into the dispatch mechanisms of system calls presented in the article allows us to solve the tasks of intercepting not only specific system calls, but the dispatch mechanism as a whole. The proposed approach to the implementation of loading and unloading the module allows you to correctly integrate into the system, and also makes it possible, importantly, to ensure the operability of the system after unloading. Active use of the disassembler allows you to reliably solve the problem of finding hidden and non-exported characters.
Traditionally, the kernel module code that implements the actions necessary to intercept functions is available on github .
This article is a continuation of the previously announced cycle devoted to particular issues of the implementation of imposed security features, and, in particular, integration into software systems.
I. Embedding approaches
There are various ways to intercept system calls to the Linux kernel. First of all, it is worth noting that to intercept single system calls, the previously considered method of intercepting kernel functions can be used . Indeed, since most system calls are represented by the corresponding functions (for example, sys_open ), the task of intercepting them is equivalent to the task of intercepting these functions. However, with the increase in the number of intercepted system calls and the complexity of the "business logic", this approach may be limited.
A more universal way is to modify entries in the system call tables (more on the tables below), which contain pointers to functions that implement the logic of a system call. Tables are used by the kernel for scheduling, when a pointer to a handler function is selected from the corresponding table by the number of the system call requested by the application with its subsequent execution. Replacing such a pointer will allow you to change the logic of the kernel in terms of handling system calls. Looking ahead, it is worth noting that for the successful implementation of this method, the tables themselves will need to be somehow found, because they are not exported. Ultimately, intercepting a system call will consist of simply overriding the table element.
The most universal way of intercepting system calls was and remains to be the modification of the code of the system call manager so that pre- and post-processing of the context of the thread requesting any system service is provided. This option gives greater flexibility in comparison with the previous ones, as Introduces single points of state monitoring before and after the handler function.
Next, an example will be discussed in detail how to embed Linux kernel system calls into the interface by modifying the dispatcher code.
II. Dispatching system calls in the Linux kernel
Dispatching system calls is a rather complicated process with many nuances, however, many details will be omitted in this article, because, with the exception of the dispatching process itself (fetching and executing the function corresponding to the system call), you don’t need to know anything else to implement .
Traditionally, the Linux kernel supports the following system call capabilities for the x86 architecture:
- INT 80h instruction (32-bit interface, native call or emulation);
- SYSENTER instruction (32-bit interface, native call or emulation);
- SYSCALL instruction (64-bit interface, native call or emulation).
The following is an excellent illustration of a system call I borrowed, depending on the option used:
As you can see, 32-bit applications make system calls using the INT 80h and SYSENTER mechanisms, while 64-bit applications use SYSCALL. At the same time, there is support for the ability to execute 32-bit code in a 64-bit environment (the so-called compatibility mode - emulation / compatibility mode; kernel option
CONFIG_IA32_EMULATION). In this regard, there are 2 non-exportable tables in the kernel - sys_call_tableand ia32_sys_call_table(available only for emulation mode) containing the addresses of functions - system call handlers.In the general case, when all possible mechanisms are represented in a 64-bit kernel, there are 4 entry points that determine what the logic of the corresponding dispatcher will be:
- emulation INT 80h, ia32_syscall (x86 / ia32 / ia32entry.S);
- SYSENTER emulation, ia32_sysenter_target (x86 / ia32 / ia32entry.S);
- SYSCALL32 emulation, ia32_cstar_target (x86 / ia32 / ia32entry.S);
- SYSCALL, system_call (x86 / kernel / entry_64.S).
One way or another, when the application makes a system call, the kernel gains control. The system call manager for each of the considered cases has differences from the others, however, without loss of generality, their general structure can be considered using the system_call example :
0xffffffff81731670 <+0>: swapgs
0xffffffff81731673 <+3>: mov %rsp,%gs:0xc000
0xffffffff8173167c <+12>: mov %gs:0xc830,%rsp
0xffffffff81731685 <+21>: sti
0xffffffff81731686 <+22>: data32 data32 xchg %ax,%ax
0xffffffff8173168a <+26>: data32 xchg %ax,%ax
0xffffffff8173168d <+29>: sub $0x50,%rsp
0xffffffff81731691 <+33>: mov %rdi,0x40(%rsp)
0xffffffff81731696 <+38>: mov %rsi,0x38(%rsp)
0xffffffff8173169b <+43>: mov %rdx,0x30(%rsp)
0xffffffff817316a0 <+48>: mov %rax,0x20(%rsp)
0xffffffff817316a5 <+53>: mov %r8,0x18(%rsp)
0xffffffff817316aa <+58>: mov %r9,0x10(%rsp)
0xffffffff817316af <+63>: mov %r10,0x8(%rsp)
0xffffffff817316b4 <+68>: mov %r11,(%rsp)
0xffffffff817316b8 <+72>: mov %rax,0x48(%rsp)
0xffffffff817316bd <+77>: mov %rcx,0x50(%rsp)
0xffffffff817316c2 <+82>: testl $0x100801d1,-0x1f78(%rsp)
0xffffffff817316cd <+93>: jne 0xffffffff8173181e
0xffffffff817316d3 <+0>: and $0xbfffffff,%eax
0xffffffff817316d8 <+5>: cmp $0x220,%eax /* <-------- cmp $__NR_syscall_max,%eax */
0xffffffff817316dd <+10>: ja 0xffffffff817317a5
0xffffffff817316e3 <+16>: mov %r10,%rcx
0xffffffff817316e6 <+19>: callq *-0x7e7fec00(,%rax,8) /* <-------- call *sys_call_table(,%rax,8) */
0xffffffff817316ed <+26>: mov %rax,0x20(%rsp)
0xffffffff817316f2 <+0>: mov $0x1008feff,%edi
0xffffffff817316f7 <+0>: cli
0xffffffff817316f8 <+1>: data32 data32 xchg %ax,%ax
0xffffffff817316fc <+5>: data32 xchg %ax,%ax
0xffffffff817316ff <+8>: mov -0x1f78(%rsp),%edx
0xffffffff81731706 <+15>: and %edi,%edx
0xffffffff81731708 <+17>: jne 0xffffffff81731745
0xffffffff8173170a <+19>: mov 0x50(%rsp),%rcx
0xffffffff8173170f <+24>: mov (%rsp),%r11
0xffffffff81731713 <+28>: mov 0x8(%rsp),%r10
0xffffffff81731718 <+33>: mov 0x10(%rsp),%r9
0xffffffff8173171d <+38>: mov 0x18(%rsp),%r8
0xffffffff81731722 <+43>: mov 0x20(%rsp),%rax
0xffffffff81731727 <+48>: mov 0x30(%rsp),%rdx
0xffffffff8173172c <+53>: mov 0x38(%rsp),%rsi
0xffffffff81731731 <+58>: mov 0x40(%rsp),%rdi
0xffffffff81731736 <+63>: mov %gs:0xc000,%rsp
0xffffffff8173173f <+72>: swapgs
0xffffffff81731742 <+75>: sysretq
As you can see, the first instruction (
swapgs) switches data structures (from user to nuclear). Next, the stack is configured, interrupts are allowed, and also on the stack a register flow context (structure pt_regs) is formed, which is necessary during processing. Returning to the listing above, special attention should be paid to the following commands: 0xffffffff817316d8 <+5>: cmp $0x220,%eax /* <-------- cmp $__NR_syscall_max,%eax */
0xffffffff817316dd <+10>: ja 0xffffffff817317a5
0xffffffff817316e3 <+16>: mov %r10,%rcx
0xffffffff817316e6 <+19>: callq *-0x7e7fec00(,%rax,8) /* <-------- call *sys_call_table(,%rax,8) */
0xffffffff817316ed <+26>: mov %rax,0x20(%rsp)
The first line checks whether the number of the requested system call (register
%rax) matches the maximum allowed value ( __NR_syscall_max). In the event that the verification is successful, the system call will be dispatched, namely, the control will go to a function that implements the corresponding logic. Thus, the key point in the process of processing system calls is the dispatch command, which is a function call (
call *sys_call_table(,%rax,8)). We will carry out further embedding by modifying this command.III. Embedding Methodology
As noted, a universal way to integrate into the dispatcher will be to modify its code in such a way as to provide the ability to control the context of the thread before it performs the function of implementing the logic of the system call (processing), as well as after its execution (post-processing).
In order to implement the embedding in the described way, it is proposed to patch the dispatcher slightly by modifying the dispatch command (
call *sys_call_table(,%rax,8)), and write the unconditional jump command ( JMP REL32) to the handler on top of it service_stub. In this case, the general structure of such a handler will look as follows (hereinafter referred to as pseudo-code):system_call:
swapgs
..
jmp service_stub /* <-------- ТУТ БЫЛ call *sys_call_table(,%rax,8) */
mov %rax,0x20(%rsp) /* <-------- СЮДА ОСУЩЕСТВЛЯЕТСЯ ВОЗВРАТ ИЗ service_stub */
...
swapgs
sysretq
service_stub:
...
call ServiceTraceEnter /* void ServiceTraceEnter(struct pt_regs *) */
...
call sys_call_table[N](args)
...
call ServiceTraceLeave(regs) /* void ServiceTraceLeave(struct pt_regs *) */
...
jmp back
Here
ServiceTraceEnter()and ServiceTraceLeave()are the functions of pre- and post-processing, respectively. Their parameters are a pointer to pt_regs- a register structure representing the context of the stream. The final instruction is the command to transfer control to the system call dispatcher code, from where the call to this handler was made earlier. The following is the code for the service_syscall64 handler , used as an example to intercept
system_call(SYSCALL instruction):.global service_syscall64
service_syscall64:
SAVE_REST
movq %rsp, %rdi
call ServiceTraceEnter
RESTORE_REST
LOAD_ARGS 0
movq %r10, %rcx
movq ORIG_RAX - ARGOFFSET(%rsp), %rax
call *0x00000000(,%rax,8) // origin call
movq %rax, RAX - ARGOFFSET(%rsp)
SAVE_REST
movq %rsp, %rdi
call ServiceTraceLeave
RESTORE_REST
movq RAX - ARGOFFSET(%rsp), %rax
jmp 0x00000000
Как видно, он имеет рассматриваемую выше структуру. Точные значения указателей и смещений настраиваются в процессе загрузки модуля (об этом будет рассказано далее). Кроме того, в приведённом фрагменте присутствуют дополнительные элементы (
SAVE_REST, RESTORE_REST, LOAD_ARGS), назначение которых в основном заключается в формировании контекста потока (pt_regs) перед вызовом функций ServiceTraceEnter и ServiceTraceLeave.IV. Особенности осуществления встраивания
Осуществление встраивания в механизмы диспетчеризации системных вызовов ядра ОС Linux так или иначе предполагает необходимость решения следующих практических задач:
- определение адресов диспетчеров системных вызовов;
- определение адресов таблиц диспетчеризации системных вызовов;
- модификация кода диспетчеров;
- настройка обработчиков;
- выгрузка модуля.
Determining the addresses of system call dispatchers
The presence of several dispatchers in the system implies the need to determine their addresses. It was noted above that each dispatcher corresponds to its own “method” of making a request for a system call. Therefore, the appropriate mechanisms will be used to determine the required addresses:
- INT 80h, reading an IDT table vector ( more );
- SYSENTER, reading the contents of the MSR register with the number MSR_IA32_SYSENTER_EIP ( more );
- SYSCALL32, reading the contents of the MSR register with the number MSR_CSTAR ( more );
- SYSCALL, reading the contents of the MSR register with the number MSR_LSTAR ( more ).
Thus, each of the required addresses is easily determined.
Determination addresses dispatch tables syscalls
As noted above, tables
sys_call_tableand ia32_sys_call_tableare not exported. There are different ways to determine their addresses, however, having determined the addresses of dispatchers in the previous step, the addresses of the tables are also determined simply by searching for a dispatch instruction of the form call sys_call_table[N]. For these purposes, it is rational to use a disassembler ( udis86) By sequentially sorting through the instructions, starting with the very first one, you can reach the desired command, the argument of which will be the address of the corresponding table. Due to the fact that the dispatcher structure is well established, it is possible to unambiguously determine the characteristics of the desired command (CALL with a length of 7 bytes) and with a high degree of reliability obtain the required table address value from it.
If for some reason this is not enough, you can strengthen the verification of the received address. To do this, for example, you can check whether the value in the cell with the number of the
__NR_openproposed table is equal to the address of the sys_open function. However, in this example, such additional checks are not performed. Modification of dispatchers code
When modifying the code of system call dispatchers, you need to consider that their code is read-only (ReadOnly). In addition, code modification on a working system should be atomic, i.e. so that during the modification process there are no undefined states when any of the threads sees a partially completed record.
In a previous article, the correct way to write to a write-protected page using the creation of temporary mappings is discussed. There is no need to repeat something here. With regard to atomicity, this issue was also discussed earlier when the topic of interception of the functions of the nucleus was considered.
Thus, it is advisable to modify the write-protected code using temporary mappings , as well as a special interface of the Linux kernel -
stop_machine. Configuring handlers
In accordance with the presented embedding method, the code of each of the dispatchers is modified in such a way that the 7-byte dispatch command
CALL MEM32is replaced with a 5-byte unconditional jump command to the corresponding handler JMP REL32. As a result, certain restrictions are imposed on the transition range. The handler should be located no more than ± 2 GB from the location of the team JMP REL32.In accordance with the structure of the handlers, they contain commands (JMP and CALL) that require specifying exact arguments (for example, return addresses or addresses of the system call table). Due to the fact that such values are not available at the stage of compilation or loading of the module, they must be affixed “manually”, after loading, before starting work.
Another important feature when setting up handlers is the need to ensure that the module can be unloaded while maintaining the system’s health. For these purposes, the handler code must remain in the system even after unloading the main module (more on this later).
Unload module
Unloading the module should be carried out while maintaining the health of the system. This means that, after the module has been unloaded, the system should function normally. This task is not trivial due to the fact that with the unloading of the module, all the code used in it is unloaded.
For example, you can imagine a situation where a thread making a system call has fallen asleep in the kernel. Until the moment when he wakes up, someone tries to unload the module. In principle, nothing prevents the system from doing this. As a result, when the thread in question wakes up and completes the requested system call, control will return to the appropriate handler (which is why it should not be unloaded).
However, not unloading the code of the handlers is not the only condition for maintaining the system's operability after unloading the module. It is worth recalling that the real system call in the handler was “wrapped” in a couple of calls to the ServiceTraceEnter and ServiceTraceLeave trace functions , the code of which was located in the unloaded module.
Therefore, in order not to get into a situation where, upon returning from a system call, the thread would try to call a function that is physically no longer there, it is necessary to re-modify the code of each handler, eliminating more invalid calls from it (in other words, clogging them with NOPs).
V. Features of the implementation of the kernel module
The following describes the structure of the kernel module that implements the Linux kernel kernel system call dispatch mechanism.
The key structure of the module is struct scentry - a structure containing the information necessary for embedding in the appropriate dispatcher. The structure contains the following fields:
typedef struct scentry {
const char *name;
const void *entry;
const void *table;
const void *pcall;
void *pcall_map;
void *stub;
const void *handler;
void (*prepare)(struct scentry *);
void (*implant)(struct scentry *);
void (*restore)(struct scentry *);
void (*cleanup)(struct scentry *);
} scentry_t;
Structures are combined into an array that determines how and with what parameters to embed:
scentry_t elist[] = {
...
{
.name = "system_call", /* SYSCALL: MSR(LSTAR), kernel/entry_64.S (1) */
.handler = service_syscall64,
.prepare = prepare_syscall64_1
},
{
.name = "system_call", /* SYSCALL: MSR(LSTAR), kernel/entry_64.S (2) */
.handler = service_syscall64,
.prepare = prepare_syscall64_2
},
...
};
With the exception of the indicated fields, the remaining elements of the structure are filled in automatically - the function is responsible for this
prepare. The following is an example implementation of a function to prepare for embedding into the dispatcher of the SYSCALL command:extern void service_syscall64(void);
static void prepare_syscall64_1(scentry_t *se)
{
/*
* searching for -- 'call *sys_call_table(,%rax,8)'
* http://lxr.free-electrons.com/source/arch/x86/kernel/entry_64.S?v=3.13#L629
*/
se->entry = get_symbol_address(se->name);
se->entry = se->entry ? se->entry : to_ptr(x86_get_msr(MSR_LSTAR));
if (!se->entry) return;
se->pcall = ud_find_insn(se->entry, 512, UD_Icall, 7);
if (!se->pcall) return;
se->table = to_ptr(*(int *)(se->pcall + 3));
}
As you can see, first of all, an attempt is made to resolve the symbol name to its corresponding address (
se->entry). If the address cannot be determined in this way, the mechanisms specific to each dispatcher come into play (in this case, reading the MSR register with the number MSR_LSTAR). Next, for the dispatcher found, the dispatch command (
se->pcall) is searched and, if successful, the address of the system call table used by the dispatcher is determined. The completion of the preparation phase is the creation of the handler code used by the dispatcher after its modification. Below is the stub_fixup function that does this:
static void fixup_stub(scentry_t *se)
{
ud_t ud;
memset(se->stub, 0x90, STUB_SIZE);
ud_initialize(&ud, BITS_PER_LONG, \
UD_VENDOR_ANY, se->handler, STUB_SIZE);
while (ud_disassemble(&ud)) {
void *insn = se->stub + ud_insn_off(&ud);
const void *orig_insn = se->handler + ud_insn_off(&ud);
memcpy(insn, orig_insn, ud_insn_len(&ud));
/* fixup sys_call_table dispatcher calls (FF.14.x5.xx.xx.xx.xx) */
if (ud.mnemonic == UD_Icall && ud_insn_len(&ud) == 7) {
x86_insert_call(insn, NULL, se->table, 7);
continue;
}
/* fixup ServiceTraceEnter/Leave calls (E8.xx.xx.xx.xx) */
if (ud.mnemonic == UD_Icall && ud_insn_len(&ud) == 5) {
x86_insert_call(insn, insn, orig_insn + (long)(*(int *)(orig_insn + 1)) + 5, 5);
continue;
}
/* fixup jump back (E9.xx.xx.xx.xx) */
if (ud.mnemonic == UD_Ijmp && ud_insn_len(&ud) == 5) {
x86_insert_jmp(insn, insn, se->pcall + 7);
break;
}
}
se->pcall_map = map_writable(se->pcall, 64);
}
As you can see, the main role of this function is to create a copy of the handlers and then configure them to current addresses. Disassembler is also actively used here. The simple structure of the handlers allows you to do without any complex logic. A signal to exit the loop is the detection of a command
JMP REL32that returns control to the dispatcher. The preparation phase is followed by the phase of implanting the code into the kernel code. This phase is quite simple and consists of writing one single instruction (
JMP REL32) in the code of each of the system services. When the module is unloaded, the restore phase is first executed , which consists in restoring the code of the system call dispatchers, as well as modifying the code of the handlers:
static void generic_restore(scentry_t *se)
{
ud_t ud;
if (!se->pcall_map) return;
ud_initialize(&ud, BITS_PER_LONG, \
UD_VENDOR_ANY, se->stub, STUB_SIZE);
while (ud_disassemble(&ud)) {
if (ud.mnemonic == UD_Icall && ud_insn_len(&ud) == 5) {
memset(se->stub + ud_insn_off(&ud), 0x90, ud_insn_len(&ud));
continue;
}
if (ud.mnemonic == UD_Ijmp)
break;
}
debug(" [o] restoring original call instruction %p (%s)\n", se->pcall, se->name);
x86_insert_call(se->pcall_map, NULL, se->table, 7);
}
As you can see, in the handler code, all found 5-byte CALL commands will be replaced by a sequence of NOPs, which will exclude attempts to execute non-existent code upon returning from a system call. This was discussed earlier.
The functions corresponding to the implant and restore phases are performed in the context
stop_machine, so all mappings used must be prepared in advance. The final phase during unloading is the clenup phase , where internal resources (
pcall_map) are freed . It is worth noting once again that after the module is unloaded , the area containing the code for the handlers will always remain in the kernel memory . As noted earlier, this is a prerequisite for the correct operation of the system after unloading the module.
Thus, the basic principles of embedding the kernel system calls into the mechanism of system calls are analyzed using an example, and the possibility of intercepting them is illustrated.
VI. Testing and debugging
For testing purposes, we intercept a system call
open(2). Below is the trace_syscall_entry function that implements this interception using the ServiceTraceEnter handler :static void trace_syscall_entry(int arch, unsigned long major, \
unsigned long a0, unsigned long a1, unsigned long a2, unsigned long a3)
{
char *filename = NULL;
if (major == __NR_open || major == __NR_ia32_open) {
filename = kmalloc(PATH_MAX, GFP_KERNEL);
if (!filename || strncpy_from_user(filename, (const void __user *)a0, PATH_MAX) < 0)
goto out;
printk("%s open(%s) [%s]\n", arch ? "X86_64" : "I386", filename, current->comm);
}
out:
if (filename) kfree(filename);
}
void ServiceTraceEnter(struct pt_regs *regs)
{
if (IS_IA32)
trace_syscall_entry(0, regs->orig_ax, \
regs->bx, regs->cx, regs->dx, regs->si);
#ifdef CONFIG_X86_64
else
trace_syscall_entry(1, regs->orig_ax, \
regs->di, regs->si, regs->dx, regs->r10);
#endif
}
Assembly and loading of the module is carried out by standard means:
$ git clone https://github.com/milabs/kmod_hooking_sct
$ cd kmod_hooking_sct
$ make
$ sudo insmod scthook.ko
As a result
dmesg, the following information should appear in the kernel log (command ):[ 5217.779766] [scthook] # SYSCALL hooking module
[ 5217.780132] [scthook] # prepare
[ 5217.785853] [scthook] [o] prepared stub ffffffffa000c000 (ia32_syscall)
[ 5217.785856] [scthook] entry:ffffffff81731e30 pcall:ffffffff81731e92 table:ffffffff81809cc0
[ 5217.790482] [scthook] [o] prepared stub ffffffffa000c200 (ia32_sysenter_target)
[ 5217.790484] [scthook] entry:ffffffff817319a0 pcall:ffffffff81731a36 table:ffffffff81809cc0
[ 5217.794931] [scthook] [o] prepared stub ffffffffa000c400 (ia32_cstar_target)
[ 5217.794933] [scthook] entry:ffffffff81731be0 pcall:ffffffff81731c75 table:ffffffff81809cc0
[ 5217.797517] [scthook] [o] prepared stub ffffffffa000c600 (system_call)
[ 5217.797518] [scthook] entry:ffffffff8172fcb0 pcall:ffffffff8172fd26 table:ffffffff81801400
[ 5217.800013] [scthook] [o] prepared stub ffffffffa000c800 (system_call)
[ 5217.800014] [scthook] entry:ffffffff8172fcb0 pcall:ffffffff8172ff38 table:ffffffff81801400
[ 5217.800014] [scthook] # prepare OK
[ 5217.800015] [scthook] # implant
[ 5217.800052] [scthook] [o] implanting jump to stub handler ffffffffa000c000 (ia32_syscall)
[ 5217.800054] [scthook] [o] implanting jump to stub handler ffffffffa000c200 (ia32_sysenter_target)
[ 5217.800054] [scthook] [o] implanting jump to stub handler ffffffffa000c400 (ia32_cstar_target)
[ 5217.800055] [scthook] [o] implanting jump to stub handler ffffffffa000c600 (system_call)
[ 5217.800056] [scthook] [o] implanting jump to stub handler ffffffffa000c800 (system_call)
[ 5217.800058] [scthook] # implant OK
Correct development of the interception
open(2)will lead to the appearance of messages in the same log as:[ 5370.999929] X86_64 open(/usr/share/locale-langpack/en_US.utf8/LC_MESSAGES/libc.mo) [perl]
[ 5370.999930] X86_64 open(/usr/share/locale-langpack/en_US/LC_MESSAGES/libc.mo) [perl]
[ 5370.999932] X86_64 open(/usr/share/locale-langpack/en.UTF-8/LC_MESSAGES/libc.mo) [perl]
[ 5370.999934] X86_64 open(/usr/share/locale-langpack/en.utf8/LC_MESSAGES/libc.mo) [perl]
[ 5370.999936] X86_64 open(/usr/share/locale-langpack/en/LC_MESSAGES/libc.mo) [perl]
[ 5371.001308] X86_64 open(/etc/login.defs) [cron]
[ 5372.422399] X86_64 open(/home/ilya/.cache/awesome/history) [awesome]
[ 5372.424013] X86_64 open(/dev/null) [awesome]
[ 5372.424682] I386 open(/etc/ld.so.cache) [skype]
[ 5372.424714] I386 open(/usr/lib/i386-linux-gnu/libXv.so.1) [skype]
[ 5372.424753] I386 open(/usr/lib/i386-linux-gnu/libXss.so.1) [skype]
[ 5372.424789] I386 open(/lib/i386-linux-gnu/librt.so.1) [skype]
[ 5372.424827] I386 open(/lib/i386-linux-gnu/libdl.so.2) [skype]
[ 5372.424856] I386 open(/usr/lib/i386-linux-gnu/libX11.so.6) [skype]
[ 5372.424896] I386 open(/usr/lib/i386-linux-gnu/libXext.so.6) [skype]
[ 5372.424929] I386 open(/usr/lib/i386-linux-gnu/libQtDBus.so.4) [skype]
[ 5372.424961] I386 open(/usr/lib/i386-linux-gnu/libQtWebKit.so.4) [skype]
[ 5372.425003] I386 open(/usr/lib/i386-linux-gnu/libQtXml.so.4) [skype]
[ 5372.425035] I386 open(/usr/lib/i386-linux-gnu/libQtGui.so.4) [skype]
[ 5372.425072] I386 open(/usr/lib/i386-linux-gnu/libQtNetwork.so.4) [skype]
[ 5372.425103] I386 open(/usr/lib/i386-linux-gnu/libQtCore.so.4) [skype]
[ 5372.425151] I386 open(/lib/i386-linux-gnu/libpthread.so.0) [skype]
[ 5372.425191] I386 open(/usr/lib/i386-linux-gnu/libstdc++.so.6) [skype]
[ 5372.425233] I386 open(/lib/i386-linux-gnu/libm.so.6) [skype]
[ 5372.425265] I386 open(/lib/i386-linux-gnu/libgcc_s.so.1) [skype]
[ 5372.425292] I386 open(/lib/i386-linux-gnu/libc.so.6) [skype]
[ 5372.425338] I386 open(/usr/lib/i386-linux-gnu/libxcb.so.1) [skype]
[ 5372.425380] I386 open(/lib/i386-linux-gnu/libdbus-1.so.3) [skype]
[ 5372.425416] I386 open(/lib/i386-linux-gnu/libz.so.1) [skype]
[ 5372.425444] I386 open(/usr/lib/i386-linux-gnu/libXrender.so.1) [skype]
[ 5372.425475] I386 open(/usr/lib/i386-linux-gnu/libjpeg.so.8) [skype]
[ 5372.425510] I386 open(/lib/i386-linux-gnu/libpng12.so.0) [skype]
[ 5372.425546] I386 open(/usr/lib/i386-linux-gnu/libxslt.so.1) [skype]
[ 5372.425579] I386 open(/usr/lib/i386-linux-gnu/libxml2.so.2) [skype]
Moreover, it is worth noting that the interception operation for 32-bit applications (for example, Skype) is also carried out correctly, which is confirmed by the presence of messages starting with I386, and not with X86_64. Thus, the example
open(2)illustrates the possibility of intercepting system calls.VII. Conclusion
The method of embedding the Linux OS kernel into the dispatch mechanisms of system calls presented in the article allows us to solve the tasks of intercepting not only specific system calls, but the dispatch mechanism as a whole. The proposed approach to the implementation of loading and unloading the module allows you to correctly integrate into the system, and also makes it possible, importantly, to ensure the operability of the system after unloading. Active use of the disassembler allows you to reliably solve the problem of finding hidden and non-exported characters.
Traditionally, the kernel module code that implements the actions necessary to intercept functions is available on github .