Speeding up process and thread enumeration in Windows

Sometimes you need to list all the processes or threads that are currently running on Windows. This may be needed for various reasons. Perhaps we are writing a system utility like Process Hacker , or maybe we want to somehow respond to the start / stop of new processes or threads (write a log, check them, inject our own code into them). The most correct way to implement this is, of course, writing a driver. Everything is simple there - we use PsSetCreateProcessNotifyRoutine and PsSetCreateThreadNotifyRoutineto set callback functions that will be called when starting / stopping processes and threads. It works very fast and does not eat resources. That's what all serious tools do. But developing drivers is not always the right way. They need to be able to write correctly, they recently have to be sure to sign with certificates (which is not free) and register with Microsoft (which is not fast). And yet they are not convenient to distribute - for example, programs with them can not be uploaded to the Microsoft Store.

Well then, let's take advantage of what public WinAPI offers. And he offers the CreateToolhelp32Snapshot () function , which is proposed to be used somehow like this. Everything seems to be good - there is information about processes, flows. A little frustrating is the fact that instead of elegant callbacks, we are forced to do endless pooling in a loop, but that’s okay.

The biggest problem here is performance. A bunch of CreateToolhelp32Snapshot () + Process32First () + Process32Next () works very slowly. Perhaps the problem lies somewhere in the same area as the problem described here in this article with Heap32First () + Heap32Next (). Briefly - for historical reasons, in some places a linear list passage takes quadratic time.

Is it possible to somehow accelerate all this? Can. But you have to get off the bright path of using WinAPI's public functions alone.

For a long time I thought that WinAPI functions are divided into public (described in MSDN, valid for use) and undocumented (found during reverse engineering of system libraries that are not described in MSDN, not officially supported). This black and white world seemed simple and logical to me: we use public functions for public products, for personal training purposes, utilities, internal tools - you can try to use undocumented ones as well.

But it turned out that there is a gray zone between these worlds. These are functions that are described in MSDN (this makes them look like public), but Microsoft says it can change or delete them at any time (as undocumented). Why do such functions exist? It's simple - on the one hand, their benefits are too great to hide, but on the other hand, future versions of the OS may have internal reasons for changing them. Such functions are called “private” in one of the terminologies I have encountered. An example is NtQuerySystemInformation () .

Evaluate the first line in the documentation - "NtQuerySystemInformation may be altered or unavailable in future versions of Windows. Applications should use the alternate functions listed in this topic "- while for half of the described applications of these same" alternative functions "and not described. Is it possible to use such functions? Everyone decides for himself. Personally, it seems to me that "fear of wolves - do not go to the forest." As if any other function on MSDN is guaranteed to never become "altered or unavailable in future versions of Windows". Any code needs to be checked on new versions of the OS. Any code needs support and adaptation. And if there is a function that now works and benefits, then why not use it?

And NtQuerySystemInformation brings significant benefits. Here is the graph of the performance growth that the MHook library received when switching from CreateToolhelp32Snapshot () to NtQuerySystemInformation ():



How to use NtQuerySystemInformation ()? Very simple:

1. We are looking for the function "NtQuerySystemInformation" in the library "ntdll.dll". Theoretically, it may not be found there, but in practice on all versions of the OS from Vista to Win10 it is for sure. Not sure about XP (there was no possibility and need to check)
2. Allocate memory for the buffer with the results
3. We call a function with the SystemProcessInformation parameter
4. We bypass the results, moving between records for individual processes using the value "NextEntryOffset" from the description structure of the current process.
5. Do not forget to free the memory allocated in step number 2

Code examples can be found here or here .

Personally, using the transition from CreateToolhelp32Snapshot () to NtQuerySystemInformation (), I was able to win about 2% of the total processor load in one fairly demanding scenario, which is quite a lot.

The moral of this story is that the slow operation of the WinAPI function is not always the final sentence. The operating system is a big and complicated thing, it may well turn out to be another way to get the necessary information.