Synthetic symbols and modules (WinDbg / DbgEng)
In this post we will talk about synthetic modules and symbols of the Windows debugging engine (debugger engine). That is, about entities that can be artificially added to the debugger for coloring memory addresses.
Synthetic modules
In general, a module within the debugging engine (debugger engine) is a certain continuous area of virtual memory: the base address of the module (the address of the first byte of the projected file) and its size. For each type of debugging (live debugging / dump analysis, user mode / kernel mode, etc.), the debugger has its own mechanism for reading and keeping up to date the list of loaded modules. The easiest way to force the updating of the list of loaded modules is with the .reload command with the / s parameter .
If we talk, for example, about live debugging of the user-mode process, the list of loaded modules is based on three main loader lists : the module loading order list (InLoadOrderModuleList), the list of modules in memory (InMemoryOrderModuleList) and the initialization order list (InInitializationOrderModuleList). On the one hand, we ( almost ) do not interfere with taking arbitrary data (from a PE file on a disk, for example) and manually marking it in memory for execution. On the other hand, the techniques of removing a regular DLL loaded from the above-mentioned three lists of the loader (hiding the module) have long been known .
In both cases, when debugging such a situation, it will be useful to mark the area of the hidden module. For this fit synthetic modules. As a practical experiment, you can simply force WinDbg to discard the loaded module from its internal list with the same .reload command , but with the / u parameter .
Next, in the listing lists, I will use the usual debugging process of the user mode (notepad.exe). First, let's remember the base address of the ntdll module (0x07ffa8a160000) and calculate its size (0x01e1000):
0:007> lm m ntdll
Browse full module list
start end module name
00007ffa`8a160000 00007ffa`8a341000 ntdll
0:007> ? 00007ffa`8a341000-00007ffa`8a160000
Evaluate expression: 1970176 = 00000000`001e1000Consider the list of the simple function RtlFreeSid from the ntdll module, which calls the function RtlFreeHeap from this module, simultaneously remembering the addresses of the characters RtlFreeSid and RtlFreeHeap (0x7ffa8a1cbc20 and 0x7ffa8a176df0):
0:007> uf ntdll!RtlFreeSid
ntdll!RtlFreeSid:
00007ffa`8a1cbc20 4053 push rbx
00007ffa`8a1cbc22 4883ec20 sub rsp,20h
00007ffa`8a1cbc26 488bd9 mov rbx,rcx
00007ffa`8a1cbc29 33d2 xor edx,edx
00007ffa`8a1cbc2b 65488b0c2560000000 mov rcx,qword ptr gs:[60h]
00007ffa`8a1cbc34 4c8bc3 mov r8,rbx
00007ffa`8a1cbc37 488b4930 mov rcx,qword ptr [rcx+30h]
00007ffa`8a1cbc3b e8b0b1faff call ntdll!RtlFreeHeap (00007ffa`8a176df0)
00007ffa`8a1cbc40 33c9 xor ecx,ecx
00007ffa`8a1cbc42 85c0 test eax,eax
00007ffa`8a1cbc44 480f45d9 cmovne rbx,rcx
00007ffa`8a1cbc48 488bc3 mov rax,rbx
00007ffa`8a1cbc4b 4883c420 add rsp,20h
00007ffa`8a1cbc4f 5b pop rbx
00007ffa`8a1cbc50 c3 retAlso, according to the above listing, it is easy to calculate the size of the function ntdll! RtlFreeSid:
0:007> ? 00007ffa`8a1cbc51 - 0x07ffa8a1cbc20
Evaluate expression: 49 = 00000000`00000031And the size of the function ntdll! RtlFreeHeap is not important for our experiment, so for simplicity we can take it equal to one byte.
Now simulate hiding the ntdll module:
0:007> .reload /u ntdll
Unloaded ntdllThe field of this work with the address of the function ntdll! RtlFreeSid in the debugger is not so informative (and you can already access the beginning of the function only at the virtual address):
0:007> uf 00007ffa`8a1cbc20
00007ffa`8a1cbc20 4053 push rbx
00007ffa`8a1cbc22 4883ec20 sub rsp,20h
00007ffa`8a1cbc26 488bd9 mov rbx,rcx
00007ffa`8a1cbc29 33d2 xor edx,edx
00007ffa`8a1cbc2b 65488b0c2560000000 mov rcx,qword ptr gs:[60h]
00007ffa`8a1cbc34 4c8bc3 mov r8,rbx
00007ffa`8a1cbc37 488b4930 mov rcx,qword ptr [rcx+30h]
00007ffa`8a1cbc3b e8b0b1faff call 00007ffa`8a176df0
00007ffa`8a1cbc40 33c9 xor ecx,ecx
00007ffa`8a1cbc42 85c0 test eax,eax
00007ffa`8a1cbc44 480f45d9 cmovne rbx,rcx
00007ffa`8a1cbc48 488bc3 mov rax,rbx
00007ffa`8a1cbc4b 4883c420 add rsp,20h
00007ffa`8a1cbc4f 5b pop rbx
00007ffa`8a1cbc50 c3 retIn order to add a synthetic module, you need to call the program interface. IDebugSymbols3 :: AddSyntheticModule .There are simply no built-in commands that would allow you to make this call (as far as I know).Mikhail I. Izmestev suggested that there is a similar mechanism - the same .reload , but with the name, address and size parameters: "Module = Address, Size" (and, perhaps, it is implemented on synthetic modules).
0:007> .reload ntdll=00007ff8`470e1000,001e1000
*** WARNING: Unable to verify timestamp for ntdll
*** ERROR: Module load completed but symbols could not be loaded for ntdll
0:007> uf 00007ff8`4714bc20
ntdll+0x6ac20:
00007ff8`4714bc20 4053 push rbx
00007ff8`4714bc22 4883ec20 sub rsp,20h
00007ff8`4714bc26 488bd9 mov rbx,rcx
00007ff8`4714bc29 33d2 xor edx,edx
00007ff8`4714bc2b 65488b0c2560000000 mov rcx,qword ptr gs:[60h]
00007ff8`4714bc34 4c8bc3 mov r8,rbx
00007ff8`4714bc37 488b4930 mov rcx,qword ptr [rcx+30h]
00007ff8`4714bc3b e8b0b1faff call ntdll+0x15df0 (00007ff8`470f6df0)
00007ff8`4714bc40 33c9 xor ecx,ecx
00007ff8`4714bc42 85c0 test eax,eax
00007ff8`4714bc44 480f45d9 cmovne rbx,rcx
00007ff8`4714bc48 488bc3 mov rax,rbx
00007ff8`4714bc4b 4883c420 add rsp,20h
00007ff8`4714bc4f 5b pop rbx
00007ff8`4714bc50 c3 retWell, we'll use the debugger extension, and here comes pykd
. In the most recent (at the time of writing) release 0.3.4.3 , functions for managing synthetic modules were added: addSyntheticModule (...) (which is needed in our case) and removeSyntheticModule (...).
According to the previously memorized base address and module size, we add information about the hidden ntdll module to the debugger (if an error occurs, an exception will be thrown, so silent execution is a sign of success, printing warnings can be ignored):
0:007> !py
Python 2.7.15 (v2.7.15:ca079a3ea3, Apr 30 2018, 16:30:26) [MSC v.1500 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
(InteractiveConsole)
>>> addSyntheticModule(0x07ffa8a160000, 0x01e1000, 'ntdll')
*** WARNING: Unable to verify timestamp for ntdll
>>> exit()Now the disassembler listing has become a bit more informative:
0:007> uf 00007ffa`8a1cbc20
ntdll+0x6bc20:
00007ffa`8a1cbc20 4053 push rbx
00007ffa`8a1cbc22 4883ec20 sub rsp,20h
00007ffa`8a1cbc26 488bd9 mov rbx,rcx
00007ffa`8a1cbc29 33d2 xor edx,edx
00007ffa`8a1cbc2b 65488b0c2560000000 mov rcx,qword ptr gs:[60h]
00007ffa`8a1cbc34 4c8bc3 mov r8,rbx
00007ffa`8a1cbc37 488b4930 mov rcx,qword ptr [rcx+30h]
00007ffa`8a1cbc3b e8b0b1faff call ntdll+0x16df0 (00007ffa`8a176df0)
00007ffa`8a1cbc40 33c9 xor ecx,ecx
00007ffa`8a1cbc42 85c0 test eax,eax
00007ffa`8a1cbc44 480f45d9 cmovne rbx,rcx
00007ffa`8a1cbc48 488bc3 mov rax,rbx
00007ffa`8a1cbc4b 4883c420 add rsp,20h
00007ffa`8a1cbc4f 5b pop rbx
00007ffa`8a1cbc50 c3 retOr rather, we see that a function inside the ntdll module calls another function inside the same module.
Synthetic characters
Listing, after adding a synthetic module, has become more informative, but still not as eloquent as the original one. It lacks characters (in our case, the names of the functions RtlFreeSid and RtlFreeHeap). To fix this again, a call is required, but another software interface - IDebugSymbols3 :: AddSyntheticSymbol . Again, pykd is ready to help us with this:
0:007> !py
Python 2.7.15 (v2.7.15:ca079a3ea3, Apr 30 2018, 16:30:26) [MSC v.1500 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
(InteractiveConsole)
>>> addSyntheticSymbol(0x07ffa8a1cbc20, 0x31, 'RtlFreeSid')
<pykd.syntheticSymbol object at 0x0000014D47699518>
>>> addSyntheticSymbol(0x07ffa8a176df0, 1, 'RtlFreeHeap')
<pykd.syntheticSymbol object at 0x0000014D476994A8>
>>> exit()The result is very similar to that when using symbols from the pdb file ntdll:
0:007> uf 00007ffa`8a1cbc20
ntdll!RtlFreeSid:
00007ffa`8a1cbc20 4053 push rbx
00007ffa`8a1cbc22 4883ec20 sub rsp,20h
00007ffa`8a1cbc26 488bd9 mov rbx,rcx
00007ffa`8a1cbc29 33d2 xor edx,edx
00007ffa`8a1cbc2b 65488b0c2560000000 mov rcx,qword ptr gs:[60h]
00007ffa`8a1cbc34 4c8bc3 mov r8,rbx
00007ffa`8a1cbc37 488b4930 mov rcx,qword ptr [rcx+30h]
00007ffa`8a1cbc3b e8b0b1faff call ntdll!RtlFreeHeap (00007ffa`8a176df0)
00007ffa`8a1cbc40 33c9 xor ecx,ecx
00007ffa`8a1cbc42 85c0 test eax,eax
00007ffa`8a1cbc44 480f45d9 cmovne rbx,rcx
00007ffa`8a1cbc48 488bc3 mov rax,rbx
00007ffa`8a1cbc4b 4883c420 add rsp,20h
00007ffa`8a1cbc4f 5b pop rbx
00007ffa`8a1cbc50 c3 retFeatures of synthetic entities
It should be noted that the creation of synthetic symbols is allowed in any existing modules, and not only in synthetic ones. That is, you can add symbols both to modules with accessible pdb files , and to modules for which we do not have debug information files. But the characters outside the modules cannot be created. That is, to mark an arbitrary memory address outside the boundaries of existing modules, you need to create an embedding synthetic module at the beginning, and then (if necessary, the module name may be enough for coloring ) to create a synthetic symbol in it.
It is also useful to mention that synthetic modules and symbols can be removed with a careless movement of the hand:
- reloading the debug symbols of the module removes all the synthetic symbols of this module;
- rebooting all modules removes all synthetic modules.
! synexts
Although the use of pykd is convenient in the overwhelming majority of cases (especially given the presence of a bootstarpper to it ), sometimes you can get into a situation where you need to spend considerable power to use pykd. For example, I once needed to debug with the host machine, which was running Windows XP. As it turned out, for a long time now, pykd does not support XP, and I needed synthetic symbols and modules. It seemed to me that for this task it is easier to assemble a separate small extension that will solve a narrow circle of necessary tasks than to restore full XP support for pykd. As a result, a separate ! Synexts project was created .
This is a simple extension that has two exports available to the user:
- ! synexts.addsymbol - creates a synthetic symbol in any existing module;
- ! synexts.addmodule - creates a synthetic module in the internal list of the debug engine.
To demonstrate, we again imitate the hiding of the ntdll module:
0:007> .reload /u ntdll
Unloaded ntdll
0:007> uf 00007ffa`8a1cbc20
00007ffa`8a1cbc20 4053 push rbx
00007ffa`8a1cbc22 4883ec20 sub rsp,20h
00007ffa`8a1cbc26 488bd9 mov rbx,rcx
00007ffa`8a1cbc29 33d2 xor edx,edx
00007ffa`8a1cbc2b 65488b0c2560000000 mov rcx,qword ptr gs:[60h]
00007ffa`8a1cbc34 4c8bc3 mov r8,rbx
00007ffa`8a1cbc37 488b4930 mov rcx,qword ptr [rcx+30h]
00007ffa`8a1cbc3b e8b0b1faff call 00007ffa`8a176df0
00007ffa`8a1cbc40 33c9 xor ecx,ecx
00007ffa`8a1cbc42 85c0 test eax,eax
00007ffa`8a1cbc44 480f45d9 cmovne rbx,rcx
00007ffa`8a1cbc48 488bc3 mov rax,rbx
00007ffa`8a1cbc4b 4883c420 add rsp,20h
00007ffa`8a1cbc4f 5b pop rbx
00007ffa`8a1cbc50 c3 retAgain, we restore knowledge about the module and the characters in the form of synthetic entities, but using the! Synexts extension:
0:007> !synexts.addmodule ntdll C:\Windows\System32\ntdll.dll 00007ffa`8a160000 0x01e1000
*** WARNING: Unable to verify timestamp for C:\Windows\System32\ntdll.dll
Synthetic module successfully added
0:007> !synexts.addsymbol RtlFreeSid 00007ffa`8a1cbc20 31
Synthetic symbol successfully added
0:007> !synexts.addsymbol RtlFreeHeap 00007ffa`8a176df0 1
Synthetic symbol successfully addedIt is assumed that the compiled synexts.dll library (corresponding to the bit width used by WinDbg) you have already copied to the winext directory of the debugger:
0:007> .chain
Extension DLL search Path:
C:\Program Files (x86)\Windows Kits\10\Debuggers\x64\WINXP;C:\Program Files (x86)\Windows Kits\10\Debuggers\x64\winext;C:\Program Files (x86)\Windows Kits\10\Debuggers\x64\winext\arcade;C:\Program Files (x86)\Windows Kits\10\Debuggers\x64\pri;C:\Program Files (x86)\Windows Kits\10\Debuggers\x64;<…>
Extension DLL chain:
pykd.pyd: image 0.3.4.3, API 1.0.0, built Thu Jan 10 19:56:25 2019
[path: C:\Program Files (x86)\Windows Kits\10\Debuggers\x64\winext\pykd.pyd]
synexts: API 1.0.0, built Fri Jan 18 17:38:17 2019
[path: C:\Program Files (x86)\Windows Kits\10\Debuggers\x64\winext\synexts.dll]
<…>And again we see the result of adding synthetic symbols to the synthetic module:
0:007> uf 00007ffa`8a1cbc20
ntdll!RtlFreeSid:
00007ffa`8a1cbc20 4053 push rbx
00007ffa`8a1cbc22 4883ec20 sub rsp,20h
00007ffa`8a1cbc26 488bd9 mov rbx,rcx
00007ffa`8a1cbc29 33d2 xor edx,edx
00007ffa`8a1cbc2b 65488b0c2560000000 mov rcx,qword ptr gs:[60h]
00007ffa`8a1cbc34 4c8bc3 mov r8,rbx
00007ffa`8a1cbc37 488b4930 mov rcx,qword ptr [rcx+30h]
00007ffa`8a1cbc3b e8b0b1faff call ntdll!RtlFreeHeap (00007ffa`8a176df0)
00007ffa`8a1cbc40 33c9 xor ecx,ecx
00007ffa`8a1cbc42 85c0 test eax,eax
00007ffa`8a1cbc44 480f45d9 cmovne rbx,rcx
00007ffa`8a1cbc48 488bc3 mov rax,rbx
00007ffa`8a1cbc4b 4883c420 add rsp,20h
00007ffa`8a1cbc4f 5b pop rbx
00007ffa`8a1cbc50 c3 ret