We listen to user input using the “Raw Input API” to control the background application
Perhaps there are almost no people left who do not know what Ctrl + C and Ctrl + V are. More experienced users know the hotkeys of commonly used applications. There are those who use more complex combinations: for example, to control a player in the background. For developers, the implementation of such functionality usually does not cause much difficulty, because this task is widespread, and much has already been written about its solution. But what if we need to minimize the user input from the joystick or presenter, and also figure out which device the event came from? To be honest, for us this task turned out to be something new. Under the cat, we will tell how we solved it in C # in a WPF application using the " Raw Input API ".
Background
Our Jalinga Studio application is used to shoot videos, webinars and online broadcasts, and they are controlled mainly using a presenter or a joystick (why did we need this, you can read in our previous article “How we animate the presentation”) Most presenters are made to work with Power Point, so they generate regular keyboard input: F5, Page Up, Page Down, etc. WPF has a standard mechanism for working with keyboard input. We used it for the first time, until we came across a significant drawback for us. The fact is that this mechanism only works when the application is active (in the foreground), but some of our clients would like to have access to, for example, a browser or other program, which will certainly deprive us of keyboard input. At first, we tried to work around this problem by creating an additional small window in the foreground, similar to how it was done in Skype. This window displays the status of the program and several buttons for control, if it is more convenient for the user to control the mouse. This approach was not the most convenient - the control window needs to be activated. If the user forgot to switch focus, then the keyboard input from the presenter went to the current active application. For example, F5 or Page Down in the browser. In addition to all this, at some point we began to miss the buttons on the presenter, and we decided to use joysticks that the standard WPF mechanism does not support.
Search for a solution
First, we formulated the requirements for the new mechanism:
- receiving information about user input if the application is running in the background;
- the ability to work with presenters, joysticks, gamepads;
- the presence of a way to distinguish input devices.
The first thing that came to mind is hooks that can be set using the SetWindowsHookEx function . But here still the question of supporting joysticks and gamepads remains open. I’m silent about the antiviruses that can take us for a keylogger, the effect of hooks on the work of other people's programs, the creation of 32-bit and 64-bit dlls, interaction with our application from another process, and the general complexity of support.
Considered using DirectInput or XInput . DirectInput is deprecated; Microsoft recommends using XInput instead . Using them, you can get user input from joysticks even in the background, if using the SetCooperativeLevel methodset the flag Background . But XInput does not support keyboard and mouse. I still did not like the pull-use model, because of which we will have to interrogate the devices of interest to us with some periodicity.
Continued to dig further. One good friend from Parallels suggested looking towards the Raw Input API". Having analyzed the capabilities of this API, we realized that everything we need is there - working with various devices of the HID class, and the ability to receive input without being an active window, and the available ID of the device from which the event came. Limitations, too there is - if the input is carried out in the admin process, and our process is not the admin, then the input events will not come. But we don’t need it. In the worst case, you can always run our application with administrator rights.
Implementation
In general, the process of obtaining user input using the "Raw Input API" consists of the following steps:
- We register the types of devices from which we will receive input events using RegisterRawInputDevices .
- We listen to the WM_INPUT events in the window procedure .
- We parse the received events using GetRawInputData .
- We determine the type of event (RAWMOUSE, RAWKEYBOARD, RAWHID) and parse it according to its type.
All this sequence had to be implemented in C # in a WPF application. In order not to write a large number of Win wrappers of API functions on our own, it was decided to use SharpDX.RawInput .
This is what simplified C # code looks like if you use Windows.Forms:
public class RawInputListener
{
public void Init(IntPtr hWnd)
{
Device.RegisterDevice(UsagePage.Generic, UsageId.GenericGamepad,
DeviceFlags.InputSink, hWnd);
Device.RegisterDevice(UsagePage.Generic, UsageId.GenericKeyboard,
DeviceFlags.InputSink, hWnd);
Device.RawInput += OnRawInput;
Device.KeyboardInput += OnKeyboardInput;
}
public void Clear()
{
Device.RawInput -= OnRawInput;
Device.KeyboardInput -= OnKeyboardInput;
}
private void OnKeyboardInput(object sender, KeyboardInputEventArgs e)
{
}
private void OnRawInput(object sender, RawInputEventArgs e)
{
}
}
The DeviceFlags.InputSink flag is needed so that the application receives the message, even if it is not in the foreground. When using this flag, you must specify hWnd.
If you use WPF, then the OnRawInput and OnKeyboardInput methods will not be called in this way, because inside the Device class, the IMessageFilter interface from Windows.Forms is implemented . If you look at the source code for Device , you can see that HandleMessage is called in the PreFilterMessage method.
Simplified C # code if you use WPF:
public class RawInputListener
{
private const int WM_INPUT = 0x00FF;
private HwndSource _hwndSource;
public void Init(IntPtr hWnd)
{
if (_hwndSource != null)
{
return;
}
_hwndSource = HwndSource.FromHwnd(hWnd);
if (_hwndSource != null)
_hwndSource.AddHook(WndProc);
Device.RegisterDevice(UsagePage.Generic, UsageId.GenericGamepad,
DeviceFlags.InputSink, hWnd);
Device.RegisterDevice(UsagePage.Generic, UsageId.GenericKeyboard,
DeviceFlags.InputSink, hWnd);
Device.RawInput += OnRawInput;
Device.KeyboardInput += OnKeyboardInput;
}
public void Clear()
{
Device.RawInput -= OnRawInput;
Device.KeyboardInput -= OnKeyboardInput;
}
private IntPtr WndProc(IntPtr hWnd, int msg, IntPtr wParam, IntPtr lParam, ref bool handled)
{
if (msg == WM_INPUT)
{
Device.HandleMessage(lParam, hWnd);
}
return IntPtr.Zero;
}
private void OnKeyboardInput(object sender, KeyboardInputEventArgs e)
{
}
private void OnRawInput(object sender, RawInputEventArgs e)
{
}
}
To figure out which device the event came from, you can use the Device property of the RawInputEventArgs class, with which you can get a DeviceName of the form:
"\\?\HID#{00001124-0000-1000-8000-00805f9b34fb}_VID&000205ac_PID&3232&Col02#8&26f2f425&7&0001#{884b96c3-56ef-11d1-bc8c-00a0c91405dd}"
You can find the Vendor ID and Product ID in this name or use the GetRawInputDeviceInfo or HidD_GetAttributes functions . Read more about VID and PID here and here .
We now turn to the analysis of events. Everything turned out to be simple with the keyboard: information already comes in a disassembled form, described in the KeyboardInputEventArgs class. But with the gamepad everything turned out to be more complicated. An argument of the base type RawInputEventArgs comes to OnRawInput. This argument must be cast to the type HidInputEventArgs and, if it works, continue to work with it. In HidInputEventArgs there is only an array of bytes that came from the device, and the number of bytes in this array is different for different joysticks and gamepads.
Unfortunately, we were able to find a little documentation telling about how to parse this data, and even that one was usually found in a short form (even in MSDN there were continuous understatements on this issue). This project in C turned out to be the most useful . It had to be brought to a working condition at first, but it was already an excellent start. After that, it was necessary to transfer the necessary parts to C #.
The first step was to wrap the native functions to call them from C # code. Here, as always, pinvoke.net helped , something I needed to describe myself. You can read about Marshal, PInvoke, and unsafe code in C # here .
The next step is to transfer the message parsing algorithm, which boils down to the following:
- Get the PreparsedData device ( GetRawInputDeviceInfo or HidD_GetPreparsedData );
- Learn about device features ( HidP_GetCaps );
- Learn about device buttons ( HidP_GetButtonCaps )
- get the list of pressed buttons ( HidP_GetUsages ).
Below is part of the data parsing code from the gamepad, which at the output gives a list of pressed buttons:
public static class RawInputParser
{
public static bool Parse(HidInputEventArgs hidInput, out List pressedButtons)
{
var preparsedData = IntPtr.Zero;
pressedButtons = new List();
try
{
preparsedData = GetPreparsedData(hidInput.Device);
if (preparsedData == IntPtr.Zero)
return false;
HIDP_CAPS hidCaps;
CheckError(HidP_GetCaps(preparsedData, out hidCaps));
pressedButtons = GetPressedButtons(hidCaps, preparsedData, hidInput.RawData);
}
catch (Win32Exception e)
{
return false;
}
finally
{
if (preparsedData != IntPtr.Zero)
{
Marshal.FreeHGlobal(preparsedData);
}
}
return true;
}
}
PreparsedData is easier to get with GetRawInputDeviceInfo , because the desired device handle is already in RawInputEventArgs. The HidD_GetPreparsedData function does not accept this handle; it requires a handle that can be obtained using CreateFile .
To get the values of the discrete buttons, the procedure is the same, only instead of HidP_GetButtonCaps you must first call HidP_GetValueCaps , and then HidP_GetUsageValue to get the discrete value of the button.
Now that the byte set from HidInputEventArgs is converted to data about which buttons are pressed, you can make a mechanism similar to what is in WPF for working with the keyboard and mouse.
The full code of the application that parses user input from gamepads and keyboards can be viewed in the RawInputWPF project on GitHub.
Total
In this way, using the “Raw Input API”, you can get user input from the keyboard, mouse, joystick, gamepad or other user input device, even if your application is in the background.
And what to do with the data on the pressed buttons is up to you.
