Using Leap Movement

  • Commodity
  • 7 minutes to read

A Leap Motion Controller is required to use this data provider.

The Leap Movement Data Provider enables articulated manus tracking for VR and could exist useful for rapid prototyping in the editor. The data provider can be configured to use the Leap Motion Controller mounted on a headset or placed on a desk face up upwardly.

LeapMotionIntroGif

This provider can exist used in editor and on device while on the Standalone platform. It tin can besides be used in editor while on the UWP platform but Non in a UWP build.

MRTK Version Leap Movement Unity Modules Versions Supported
2.6.x 4.five.0, 4.five.one
2.vii.0 - two.7.ii 4.5.0, 4.5.i, iv.half-dozen.0, 4.7.0, 4.seven.i, 4.8.0
2.7.3 4.v.0, 4.v.one, 4.half-dozen.0, 4.7.0, 4.vii.one, 4.eight.0, 4.ix.one (see notation below)

Annotation

Unity Modules four.nine.1 require the Gemini 5.2 tracking service or higher. The Unity plugin V4.9.1 is simply officially supported on Unity 2019. See release notes for four.9.i.

The Unity Modules were renamed the The Unity Plugin in V5.0.0.

MRTK Version Ultraleap Unity Plugin Versions Supported
2.vii.4 5.3.0 (see notation beneath)

Note

The Unity Plugin versions v.0.0, 5.i.0 and v.two.0 are non supported in any version of MRTK. Users are recommended to upgrade to five.three.0. Unity Plugin v.3.0 requires the Gemini v.2 tracking service or higher.

Using Spring Motion (by Ultraleap) hand tracking in MRTK

  1. Importing MRTK and the Bound Movement Unity Modules / Unity Plugin

    • Install the latest Bound Motion Tracking Software to get the Tracking Service if information technology is not already installed
    • Import the Microsoft.MixedReality.Toolkit.Foundation packet into the Unity project.
    • Download and import the latest version of the Jump Motion Unity Modules / Unity Plugin into the project
      • For Unity Modules: just import the Core bundle inside the project
      • For Unity Plugin: only import the Tracking bundle within the projection. But .unitypackages are supported for at present.

    Important

    Unity deprecated SelectionMode.OnlyUserModifiable later on Unity 2019.4.19 which results in the following error coming from the Ultraleap Hotkeys.cs script: Assets\Plugins\LeapMotion\Core\Editor\Hotkeys.cs(24,91): fault CS0619: 'SelectionMode.OnlyUserModifiable' is obsolete: ''OnlyUserModifiable' is obsolete. Use 'Editable' instead. (UnityUpgradeable) -> Editable'.

    To resolve the bug, change the SelectionMode.OnlyUserModifiable to SelectionMode.Editable in the Hotkeys.cs script located in Plugins/LeapMotion/Core/Editor/Hotkeys.cs.

  2. Integrate the Leap Motion Unity Modules / Unity Plugin with MRTK

    • After the Unity Modules / Unity Plugin are in the project, navigate to Mixed Reality Toolkit > Leap Movement > Integrate Spring Motion Unity Modules

    Notation

    Integrating the Unity Modules to MRTK adds 10 associates definitions to the project and adds references to the Microsoft.MixedReality.Toolkit.Providers.LeapMotion assembly definition. Make sure Visual Studio is airtight.

    LeapMotionIntegration

  3. Adding the Bound Motion Data Provider

    • Create a new Unity scene
    • Add MRTK to the scene past navigating to Mixed Reality Toolkit > Add together to Scene and Configure
    • Select the MixedRealityToolkit game object in the hierarchy and select Copy and Customize to clone the default mixed reality profile.

    LeapMotionProfileClone

    • Select the Input Configuration Profile

    Input Configuration Profile 1

    • Select Clone in the input system profile to enable modification.

    LeapMotionInputProfileClone

    • Open the Input Data Providers section, select Add Information Provider at the top, a new data provider volition exist added at the end of the list. Open the new data provider and ready the Type to Microsoft.MixedReality.Toolkit.LeapMotion.Input > LeapMotionDeviceManager

    Leap Add Data Provider

    • Select Clone to change the default Leap Motion settings.

    LeapDataProviderPreClone

    • The Jump Motion Information Provider contains the LeapControllerOrientation property which is the location of the Leap Motion Controller. LeapControllerOrientation.Headset indicates the controller is mounted on a headset. LeapControllerOrientation.Desk indicates the controller is placed flat on desk. The default value is ready to LeapControllerOrientation.Headset.

    • Each controller orientation contains offset properties:

      • The Headset orientation offset backdrop mirror the offset properties in the LeapXRServiceProvider component. The LeapVRDeviceOffsetMode has 3 options: Default, Manual Caput Offset and Transform. If the first mode is Default, then an offset will not be applied to the Leap Motion Controller. The Manual Caput Offset mode allows for the modification of iii backdrop: LeapVRDeviceOffsetY, LeapVRDeviceOffsetZ and LeapVRDeviceTiltX. The centrality beginning property values are then applied to default controller placement. The Transform offset style contains the LeapVRDeviceOrigin Transform belongings which specifies a new origin for the Leap Move Controller.

      • The Desk orientation contains the LeapControllerOffset property which defines the anchor position of the desk-bound leap hands. The first is calculated relative to the main camera position and the default value is (0,-0.2, 0.35) to make sure the hands appear in front and in view of the photographic camera.

        Note

        The offset backdrop in the profile are practical in one case when the awarding starts. To modify the values during runtime, become the Jump Motion Service Provider from the Spring Motion Device Manager:

                                    LeapMotionDeviceManager leapMotionDeviceManager = CoreServices.GetInputSystemDataProvider<LeapMotionDeviceManager>(); LeapXRServiceProvider leapXRServiceProvider = leapMotionDeviceManager.LeapMotionServiceProvider as LeapXRServiceProvider;

    • EnterPinchDistance and ExitPinchDistance are the distance thresholds for pinch/air tap gesture detection. The pinch gesture is calculated past measuring the altitude between the index finger tip and the thumb tip. To heighten an on input downwardly consequence, the default EnterPinchDistance is set to 0.02. To raise an on input up event (exiting the pinch), the default distance between the index finger tip and the thumb tip is 0.05.

    LeapControllerOrientation: Headset (Default) LeapControllerOrientation: Desk
    LeapHeadsetGif LeapDeskGif
    LeapHeadsetInspector LeapDeskInspector

  4. Testing the Spring Motility Data Provider

    • After Leap Move Information Provider has been added to the input organisation profile, press play, movement your hand in forepart of the Spring Motion Controller and you should see the joint representation of the manus.

  5. Edifice your project

    • Navigate to File > Build Settings
    • Just Standalone builds are supported if using the Leap Motion Data Provider.
    • For instructions on how to employ a Windows Mixed Reality headset for Standalone builds, see Building and deploying MRTK to WMR Headsets (Standalone).

Getting the paw joints

Getting joints using the Leap Motion Data Provider is identical to hand joint retrieval for an MRTK Articulated Mitt. For more information, see Hand Tracking.

With MRTK in a unity scene and the Bound Movement Data Provider added as an Input Data Provider in the Input System profile, create an empty game object and attach the following example script.

This script is a elementary case of how to retrieve the pose of the palm joint in a Leap Move Mitt. A sphere follows the left Bound hand while a cube follows the right Jump hand.

              using Microsoft.MixedReality.Toolkit; using Microsoft.MixedReality.Toolkit.Input; using Microsoft.MixedReality.Toolkit.Utilities; using Organisation.Collections.Generic; using UnityEngine;  public class LeapHandJoints : MonoBehaviour, IMixedRealityHandJointHandler {     private GameObject leftHandSphere;     private GameObject rightHandCube;      private void Starting time()     {         // Annals the HandJointHandler equally a global listener         CoreServices.InputSystem.RegisterHandler<IMixedRealityHandJointHandler>(this);          // Create a sphere to follow the left manus palm position         leftHandSphere = GameObject.CreatePrimitive(PrimitiveType.Sphere);         leftHandSphere.transform.localScale = Vector3.i * 0.03f;          // Create a cube to follow the correct hand palm position         rightHandCube = GameObject.CreatePrimitive(PrimitiveType.Cube);         rightHandCube.transform.localScale = Vector3.one * 0.03f;     }      public void OnHandJointsUpdated(InputEventData<IDictionary<TrackedHandJoint, MixedRealityPose>> eventData)     {         if (eventData.Handedness == Handedness.Left)         {             Vector3 leftHandPalmPosition = eventData.InputData[TrackedHandJoint.Palm].Position;             leftHandSphere.transform.position = leftHandPalmPosition;         }          if (eventData.Handedness == Handedness.Right)         {             Vector3 rightHandPalmPosition = eventData.InputData[TrackedHandJoint.Palm].Position;             rightHandCube.transform.position = rightHandPalmPosition;         }     }

Unity editor workflow tip

Using the Leap Motion Data Provider does not require a VR headset. Changes to an MRTK app tin can be tested in the editor with the Bound easily without a headset.

The Bound Move Hands will show up in the editor, without a VR headset plugged in. If the LeapControllerOrientation is set to Headset, then the Leap Motion controller will need to be held upwards past one hand with the camera facing forward.

Note

If the camera is moved using WASD keys in the editor and the LeapControllerOrientation is Headset, the hands will not follow the camera. The easily will only follow photographic camera movement if a VR headset is plugged in while the LeapControllerOrientation is gear up Headset. The Leap easily volition follow the camera move in the editor if the LeapControllerOrientation is fix to Desk.

Removing Leap Motion from the Project

  1. Navigate to the Mixed Reality Toolkit > Leap Motion > Separate Leap Motion Unity Modules
    • Let Unity refresh as references in the Microsoft.MixedReality.Toolkit.Providers.LeapMotion.asmdef file are modified in this pace
  2. Shut Unity
  3. Close Visual Studio, if it's open up
  4. Open File Explorer and navigate to the root of the MRTK Unity projection
    • Delete the UnityProjectName/Library directory
    • Delete the UnityProjectName/Assets/Plugins/LeapMotion directory
    • Delete the UnityProjectName/Assets/Plugins/LeapMotion.meta file
  5. Reopen Unity

In Unity 2018.4, you might notice that errors still remain in the console after deleting the Library and the Leap Motion Core Assets. If errors are logged after reopening, restart Unity once more.

Common Errors

Leap Movement has not integrated with MRTK

To examination if the Bound Move Unity Modules take integrated with MRTK:

  • Navigate to Mixed Reality Toolkit > Utilities > Leap Move > Check Integration Status
    • This will display a pop upward window with a message about whether or not the Leap Motility Unity Modules have integrated with MRTK.
  • If the message says that the assets have not been integrated:
    • Make sure the Bound Movement Unity Modules are in the project
    • Brand sure that the version added is supported, see the table at the top of the page for versions supported.
    • Effort Mixed Reality Toolkit > Utilities > Spring Motion > Integrate Leap Move Unity Modules

Copying assembly Multiplayer HLAPI failed

On import of the Spring Motion Unity Cadre Assets this error might exist logged:

              Copying assembly from 'Temp/com.unity.multiplayer-hlapi.Runtime.dll' to 'Library/ScriptAssemblies/com.unity.multiplayer-hlapi.Runtime.dll' failed

Solution

  • A curt term solution is to restart Unity. See Consequence 7761 for more information.

Leap Move Example Scene

The example scene uses the DefaultLeapMotionConfiguration profile and determines if the Unity project has been configured correctly to utilize the Leap Motion Information Provider.

The case scene is contained in the Microsoft.MixedReality.Toolkit.Examples bundle in the MRTK/Examples/Demos/HandTracking/ directory.

Come across also

  • Input Providers
  • Hand Tracking