October 18, 2016 at 10:10What's New in Intel RealSense SDK 2016 R2
- Transfer
The second release of the Intel RealSense SDK (for Windows) in 2016 (2016R2) is version 10.0.26.0396. The new version has significantly improved the operation of background segmentation modules (BGS), hand tracking in pointer mode and 3D Scan module for the SR300 camera. The first two modules are already available in the "golden" version, the 3DScan module for the SR300 has moved to the beta level. For the R200, some components of the Enhanced Photography (EP) module have moved to the gold level, including EP Meaurement. At the same time, the EP Realtime Depth Enhancement and EP Tracking modules have remained at the preliminary level for now. The remaining functions of the R200 have not changed.
The minimum DCM versions for Intel RealSense cameras are increased: 2.1.24.9476 for the R200 camera, 3.1.25.1077 for the SR300 camera. F200 requires version 1.4.27.41944 or later.
Supported version of Unity increased to 5.2.3.p3 or later (version 4.1 was previously supported). Note. Unity 5 requires 64-bit libraries.
Green Screen / BGS: Insertion (fading) and additional callbacks for near and far limits and for user detection are now available. An additional module with an extended dynamic range has been added, other changes to algorithms have been implemented that increase the accuracy of creating finger models and dark hair masks, and the accuracy of detecting other artifacts. Problems with recognizing and determining the distance to objects that absorb infrared radiation, transparent, very thin and black objects, as well as very fair hair and hair whose color is close to the background color, are still preserved. Recognition of only one person at a time is supported; the quality of the mask is significantly reduced if several people appear within the range of the camera.
Cursor mode . Improved gesture recognition accuracy, including clicking, rotating (in both directions), opening and squeezing the palm. You can reset the Adaptive Point to a specific location (ResetAdaptiveByID). The range is increased to 115 cm, and the speed of movement is up to 2 m / s, while the error is fixed with a detection range of 70 cm. Note. The default power setting is changed to OFF (you can set it to On to save power by lowering frame rate by the time your hands are out of sight of the camera, but there’s a 2 second delay when your hands are back in view of the camera).
3DScan. maxTextureResolution can limit the size of the created texture, flopPreviewImage implements a mirror image (horizontal) of the preview for the SR300 camera, useMarker improves tracking of objects that are similar in shape (cylinder, ball). In maxVertices, you can limit the quality of the vertices in the output 3D model; Fixed rounding of depth values in QueryVertices (front view cameras). In addition, in SetConfiguration for head detection using the SR300 camera, the returned message is changed from PXC_STATUS_NO_ERROR to PXC_STATUS_FEATURE_UNSUPPORTED. Feature_Unsupported is returned to scan a face without reference points using front-facing cameras; to avoid this, you need to load the Face Tracking module to get the reference points or use the Variable scan mode with SetArea ().
In PersonTracking Configuration / Data added GestureConfiguration (PersonData) and ExpressionsConfiguration (PersonExpressions), as well as recognition of the location and position of the head, human orientation, the bounding box of the head. At the same time, the configuration properties field is canceled (functions should be used for configuration). Now, when tracking, you can use re-identification, and in the tracking mode of the skeleton, 6 points are tracked (not 4).
In FaceData added LANDMARK_GROUP_RIGHT (| LEFT) _EAR, RecogntionData expanded opportunities for the transfer of data on the closest match, and modified QueryNumberOfMatches QueryMatches.
The default SmoothingLevelType for face tracking has been changed from LANDMARKS_SMOOTHING_LOW to LANDMARKS_SMOOTHING_MEDIUM. Due to this, the temporary smoothing in frames is improved.
In PXC [M] Image added PIXEL_FORMAT_DEPTH_CONFIDENCE, in PXC [M] Capture added STREAM_OPTION_DEPTH_CONFIDENCE.
In PXC [M] Projection removed improper rounding depth values in API QueryVertices for forward vision cameras; SelectOption can now return a projection map without cropping.
In the PXC [M] VideoModule interface, ProcessSample, SetProjection, and SetGPUExec are expanded (to determine the use of CPU or GPU).
In EnhancedPhoto (EP)Added insert to the surface (as well as to the plane), support for multiple stickers, AddSticker (), UpdateSticker () and RemoveSticker () along with identifier numbers for tracking stickers. You can also use the preview of the depth map PreviewEnhancedDepth () in the video processing pipeline in each frame before applying. For more information on EP issues and changes, see 2016R2 (release notes) , in the R200 Known Issues section (especially regarding flickering).
In ScenePerception thread configurationsNow you can use any sensor settings (gain, exposure, etc.) for both the color image and the depth map, including Unity. IsGravitySensorSupportEnabled and IsInertialSensorSupportEnabled are also added to the SP. You can turn off EnableSceneReconstruction for the first frame; EnableSceneReconstruction is no longer forced and automatic, so you can skip embedding the first frame in a three-dimensional model and volume. DoReconstruction supports the integration of flows into a three-dimensional model, volume and voxels even before the transmission of the first frame after the creation of the module or after a reset call. StreamSurfaceVoxels thread safety has been implemented in terms of tracking: you can run ExportSurfaceVoxels in parallel with tracking.
Note. The .chm help file may not open. Use HTML or PDF files.
In the Unity (5.2.3.p3 or later with .NET 3.5) is implemented NativeTexturePlugin support for Unity Texture2D updates using Direct3D * 9 or 11 or OpenGL * 2.0. Unity XML is located in the Plugins.Managed folder. In the SP Unity sample, you can switch to the integrated rendering, which provides a performance increase of about 50% for high-definition color images and avoids stream overload when receiving tracking results. Note. If the VS libpxcclr.unity project is not building correctly, click Project> Add Link and add [Unity installation folder] /Editor/Data/Managed/UnityEngine.dll.
The minimum DCM versions for Intel RealSense cameras are increased: 2.1.24.9476 for the R200 camera, 3.1.25.1077 for the SR300 camera. F200 requires version 1.4.27.41944 or later.
Supported version of Unity increased to 5.2.3.p3 or later (version 4.1 was previously supported). Note. Unity 5 requires 64-bit libraries.
New opportunities
Green Screen / BGS: Insertion (fading) and additional callbacks for near and far limits and for user detection are now available. An additional module with an extended dynamic range has been added, other changes to algorithms have been implemented that increase the accuracy of creating finger models and dark hair masks, and the accuracy of detecting other artifacts. Problems with recognizing and determining the distance to objects that absorb infrared radiation, transparent, very thin and black objects, as well as very fair hair and hair whose color is close to the background color, are still preserved. Recognition of only one person at a time is supported; the quality of the mask is significantly reduced if several people appear within the range of the camera.
Cursor mode . Improved gesture recognition accuracy, including clicking, rotating (in both directions), opening and squeezing the palm. You can reset the Adaptive Point to a specific location (ResetAdaptiveByID). The range is increased to 115 cm, and the speed of movement is up to 2 m / s, while the error is fixed with a detection range of 70 cm. Note. The default power setting is changed to OFF (you can set it to On to save power by lowering frame rate by the time your hands are out of sight of the camera, but there’s a 2 second delay when your hands are back in view of the camera).
3DScan. maxTextureResolution can limit the size of the created texture, flopPreviewImage implements a mirror image (horizontal) of the preview for the SR300 camera, useMarker improves tracking of objects that are similar in shape (cylinder, ball). In maxVertices, you can limit the quality of the vertices in the output 3D model; Fixed rounding of depth values in QueryVertices (front view cameras). In addition, in SetConfiguration for head detection using the SR300 camera, the returned message is changed from PXC_STATUS_NO_ERROR to PXC_STATUS_FEATURE_UNSUPPORTED. Feature_Unsupported is returned to scan a face without reference points using front-facing cameras; to avoid this, you need to load the Face Tracking module to get the reference points or use the Variable scan mode with SetArea ().
In PersonTracking Configuration / Data added GestureConfiguration (PersonData) and ExpressionsConfiguration (PersonExpressions), as well as recognition of the location and position of the head, human orientation, the bounding box of the head. At the same time, the configuration properties field is canceled (functions should be used for configuration). Now, when tracking, you can use re-identification, and in the tracking mode of the skeleton, 6 points are tracked (not 4).
In FaceData added LANDMARK_GROUP_RIGHT (| LEFT) _EAR, RecogntionData expanded opportunities for the transfer of data on the closest match, and modified QueryNumberOfMatches QueryMatches.
The default SmoothingLevelType for face tracking has been changed from LANDMARKS_SMOOTHING_LOW to LANDMARKS_SMOOTHING_MEDIUM. Due to this, the temporary smoothing in frames is improved.
In PXC [M] Image added PIXEL_FORMAT_DEPTH_CONFIDENCE, in PXC [M] Capture added STREAM_OPTION_DEPTH_CONFIDENCE.
In PXC [M] Projection removed improper rounding depth values in API QueryVertices for forward vision cameras; SelectOption can now return a projection map without cropping.
In the PXC [M] VideoModule interface, ProcessSample, SetProjection, and SetGPUExec are expanded (to determine the use of CPU or GPU).
In EnhancedPhoto (EP)Added insert to the surface (as well as to the plane), support for multiple stickers, AddSticker (), UpdateSticker () and RemoveSticker () along with identifier numbers for tracking stickers. You can also use the preview of the depth map PreviewEnhancedDepth () in the video processing pipeline in each frame before applying. For more information on EP issues and changes, see 2016R2 (release notes) , in the R200 Known Issues section (especially regarding flickering).
In ScenePerception thread configurationsNow you can use any sensor settings (gain, exposure, etc.) for both the color image and the depth map, including Unity. IsGravitySensorSupportEnabled and IsInertialSensorSupportEnabled are also added to the SP. You can turn off EnableSceneReconstruction for the first frame; EnableSceneReconstruction is no longer forced and automatic, so you can skip embedding the first frame in a three-dimensional model and volume. DoReconstruction supports the integration of flows into a three-dimensional model, volume and voxels even before the transmission of the first frame after the creation of the module or after a reset call. StreamSurfaceVoxels thread safety has been implemented in terms of tracking: you can run ExportSurfaceVoxels in parallel with tracking.
Note. The .chm help file may not open. Use HTML or PDF files.
In the Unity (5.2.3.p3 or later with .NET 3.5) is implemented NativeTexturePlugin support for Unity Texture2D updates using Direct3D * 9 or 11 or OpenGL * 2.0. Unity XML is located in the Plugins.Managed folder. In the SP Unity sample, you can switch to the integrated rendering, which provides a performance increase of about 50% for high-definition color images and avoids stream overload when receiving tracking results. Note. If the VS libpxcclr.unity project is not building correctly, click Project> Add Link and add [Unity installation folder] /Editor/Data/Managed/UnityEngine.dll.
