Switching from Hand mode in Intel RealSense SDK R4 (v6.0) to Cursor mode in Intel RealSense SDK 2016 R1

After the introduction of the Intel RealSense SR300 camera and the Intel RealSense SDK 2016 R1 package, a new gesture interaction mode appeared - the Cursor mode , which is available only when using the SR300 camera. This tutorial describes the code changes needed to enable this new functionality.

Prior to the release of the Intel RealSense SDK 2016 R1, applications in which you need to control the movement of the pointer and detect click actions used the Hand mode, and gesture recognition was used to determine the clicks. This Hand mode functionality has now evolved into the new Cursor mode. Applications that use the previous functionality, after altering the code, will be able to take advantage of higher accuracy and more advanced cursor control in Cursor mode.

Please note that Cursor mode is only available for devices with the Intel RealSense SR300 camera. Intel RealSense application developers using SR300 cameras need to upgrade to Windows * 10 and use the Intel RealSense SDK version 2016 R1.

Most likely, you already have an application written for the F200 camera using the Intel RealSense SDK R4 (v6.0). How to move on and use the new Cursor mode?

Initialization of the process pipeline should be the same as in the previous version of the Intel RealSense SDK. You must create an instance of Sense Manager and verify that there are no errors in the process.

PXCSenseManager *pSenseMgr = new PXCSenseManager::CreateInstance();
if( !pSenseMgr ) {
	< continue on to creating the modes >
}

Previously, in the Hand mode on F200 cameras, to obtain something that was at least reminiscent of a pointer, it was necessary to use the Hand module and track hands in various configurations. The code could look something like this (note that the following code is for reference, it will not compile directly in the form in which it is shown here):

PXCHandModule *pHandModule;
PXCHandData *pHandData;
int confidence;
. . . <дополнительная библиотека и настройка переменных> . . . 
pxcStatus status;
if( !pSenseMgr ) {
	status = pSenseMgr->EnableHand()
	if(status == pxcStatus::PXC_STATUS_NO_ERROR) {
	// Get an instance of PXCHandModule 
handModule = pSenseMgr->QueryHand();
// Get an instance of PXCHandConfiguration 
PXCHandConfiguration handConfig = handModule
handConfig->EnableGesture("cursor_click"); 
handConfig->ApplyChanges();
	. . . <дополнительные параметры конфигурации>  . . . 
}
}

Starting with version 2016 R1, the Intel RealSense SDK has implemented a new Cursor mode, pointer actions are separated from Hand mode. This means that you must rework the old code that requested Hand mode in Sense Manager. The new code will look like this:

PXCHandCursorModule *pCursorModule; 
PXCCursorData::BodySideType bodySide;
// обратите внимание, что значений Confidence больше нет
. . . <дополнительная библиотека и настройка переменных> . . . 
pxcStatus status;
if( !pSenseMgr ) {
// Enable handcursor tracking 
status = pSenseMgr::EnableHandCursor(); 
	if(status == pxcStatus.PXC_STATUS_NO_ERROR) {
	// Get an instance of PXCCursorModule 
pCursorModule = pSenseMgr->QueryHandCursor();
// Get an instance of the cursor configuration 
PXCCursorConfiguration *pCursorConfig = CursorModule::CreateActiveConfiguration(); 
// Make configuration changes and apply them 
pCursorConfig.EnableEngagement(true); 
pCursorConfig.EnableAllGestures();
pCursorConfig.ApplyChanges(); 
	. . . <дополнительные параметры конфигурации> . . . 
}
}

For an example of basic computational cycles for synchronous and asynchronous functions, see the Intel RealSense SDK 2016 R1 documentation in the Implementing the Main Computing Cycle Cursor Module [SR300] section .

An asynchronous (recommended) approach would look like this:

class MyHandler: public PXCSenseManager::Handler {
public:
    virtual pxcStatus PXCAPI OnModuleProcessedFrame(pxcUID mid, PXCBase *module, PXCCapture::Sample *sample) {
       // check if the callback is from the hand cursor tracking module
       if (mid==PXCHandCursorModule::CUID) {
           PXCHandCursorModule *cursorModule=module->QueryInstance();
               PXCCursorData *cursorData = cursorModule->CreateOutput();
           // process cursor tracking data
       }
       // return NO_ERROR to continue, or any error to abort
       return PXC_STATUS_NO_ERROR;
    }
};
. . . <объявление SenseManager> . . .  
// Initialize and stream data
MyHandler handler; // Instantiate the handler object
// Register the handler object
pSenseMgr->Init(&handler); 
// Initiate SenseManager’s processing loop in blocking mode
// (function exits only when processing ends)
pSenseMgr ->StreamFrames(true);
// Release SenseManager resources
pSenseMgr ->Release()

So, in the Intel RealSense SDK 2016 R1, the implementation and access to the hand pointer have changed, but all the changes are uniform, which simplifies the code rework. The above code example demonstrates this simplicity: it is shown that the structure of the program during initialization, configuration, and frame-by-frame execution can remain unchanged, while the program will use the advanced features of the new Cursor mode.

Recall that the new Cursor mode is available only for systems with the SR300 camera (the camera can be built-in or connected as a separate peripheral device) and with the RealSense SDK 2016 R1 version. The ability of the code to detect the camera model and support both the F200 and SR300 cameras will be described in other tutorials.