Window hook registration

Holmes . But tell me, my friend Watson, have you ever deregistered a window hook, especially a global one?

Watson . Hmm ... what could be easier, dear Holmes.

    ::UnhookWindowsHookEx( hhookMy);

X . Do not tell, Watson, do not tell. After this call, the DLL module containing the hook function is still loaded into all the processes into which it was loaded. The system only unloads this DLL after some time. Namely, at that moment when at least one window message will pass through the message queues of all threads (having such a queue). And so for each process on the desktop.

In . Here, wow! Do you happen to be kidding me, Holmes? This is User32. One of the main parts in Win32, which in turn relies on all other technologies in Windows. The same .Net, for example. And suddenly this is not a completely deterministic behavior.

And  CreateFile, by chance, doesn’t create the file delayed too - for example, until the first write to the file? But  CreateProcess, by chance, not ...

X . No, Watson, this time I'm not joking. CreateFileand CreateProcessthis is Kernel32 - a completely different matter.

Believe me, buddy. Carrying out my investigations, I spent a lot of time studying the internal device of the User32 module and I say with confidence that the authors wrote (and support) it in a completely different quality than Kernel32.

In . Is this just another development team?

X . Yes, obviously so.

In . Perhaps this behavior is still justified. And is it really that important when the hook DLL is unloaded?

X. Imagine that our application has updated itself via the Internet and restarts. After the restart, it turns out that some processes loaded the old version of the DLL, and some new. And in this case, the old version will not be unloaded even when messages pass through the queues.

In . Horrible!

But wait a moment. If the application updated its files, it means it renamed the old version of the hook DLL or moved it somewhere to a temporary directory. Then the new version starts ... It passes to  SetWindowsHookExthe DLL, the file path of which is now different from the DLL, which, as you say, remains loaded.

You want to say that the system will create a hook that targets the wrong DLL that we pass to  SetWindowsHookEx? It can not be!

X. In those processes from which the DLL did not manage to unload, this is exactly what will happen, I swear by my tuned keyboard.

Apparently, User32 internally saves the path that the DLL had at the time the hook was first created. At the second creation, he believes that this is the same DLL and does not need to unload and reload it.

In . Hmm ... And what about all this? What if after UnhookWindowsHookExjust wait, say, a second. Window messages occur quite often.

X . This is if the user does something with the window. And, for example, minimized windows are quiet. There is no guarantee that a message will appear in each queue within a second. Or for an hour.

In . What to do? How do you order guaranteed to unload DLLs from all processes?

(Holmes was silent and smoking, looking at Watson.)

In . Will be to you, Holmes. I am sure you have already come up with the right way out of the situation. Share, please.

X . Let our hook function, upon discovering that it is being executed for the first time in a certain thread, create a file stream in a special temporary file. And right in the name of this thread will indicate the id of the thread in which it is executed. Like that:

using namespace std;
namespace k = Kernel32;
...
if( nullptr == tls_plistiterHookedThreadFileStreamHandle)  // Переменная размещена в локальной памяти нити.
{
	_listHookedThreadFileStreamHandles.push_front( k::CreateFile(
				k::FormatMessage(
							L"%1:%2!4u! %3!4u!",
							sHookedThreadListFilePath,
							k::GetCurrentThreadId(),
							k::GetCurrentProcessId()
					),
				GENERIC_WRITE,
				FILE_SHARE_READ,
				CREATE_ALWAYS,
				FILE_FLAG_DELETE_ON_CLOSE
		));
	tls_plistiterHookedThreadFileStreamHandle 
				= new list< HANDLE>::iterator( _listHookedThreadFileStreamHandles.begin());
}

Then the main EXE will be able to iterate over these threads ( FindNextStream) and send a message WM_NULLto each thread that User32 has associated with our hook. As a result, User32 will unload the DLL.

DLL, unloading, should close all descriptors from the list _listHookedThreadFileStreamHandles.

Thus, we also get a good criterion that the thing is done - success when trying to delete our temporary file.

In . Holmes, why so much trouble? There is SendMessagean argument HWND_BROADCAST.

X . SendMessagedoes not send messages to absolutely all threads that have a queue. However, perhaps this will also work. Almost always.

(Shouts.) Mrs. Hudson, serve tea, please!

The next day

In . Holmes, a gentleman, said he was SendMessage( HWND_BROADCAST)not helping, at least in his case.

X . Hm.