👐
Hand & Finger Tracking for Unity
HomepageSupportContact
  • Hand Tracking Unity Plugin
    • Overview
    • System Requirements
    • Installation
    • QuickStart
    • Supported Joints
  • Help & Support
    • Ask for help
    • Request a feature
  • Examples
    • Detect hand and retrieve joints
    • No Code: Detect and visualize hands from a Sprite
    • Detect and visualize hands from a Sprite
    • Detect and visualize hands from a video
    • Use webcam to detect and visualize hands (2D Canvas)
    • Use webcam to detect and visualize hands (3D World space)
  • API Reference
    • Finger
      • Index
      • Middle
      • Palm
      • Pinky
      • Ring
      • Thumb
    • FingerJoint
    • FingerJointType
    • Hand
    • HandExtensions
    • HandSide
    • HandTracker
  • UI Reference
    • HandManager
    • HandViewer
    • HandVisual
    • ImageView
    • StreamSource
      • SpriteSource
      • VideoSource
      • WebcamSource
    • HandTrackerExtensions
    • WebcamSelector
Powered by GitBook
On this page
  • Step 1: Load image data
  • Image file
  • Video file
  • Camera feed
  • Step 2: Detect hands
  • Step 3: Access hand & finger data
  • Examples
  1. Hand Tracking Unity Plugin

QuickStart

A QuickStart overview on how to detect hands and retrieve finger joint information.

PreviousInstallationNextSupported Joints

Last updated 1 year ago

Following the steps described in the section, here is how to detect hands and fingers in 3 easy steps:

Step 1: Load image data

Load an image to a Texture2D. The image could be loaded from one of the following sources:

Image file

An image file is a PNG, JPG, WEBP, or BMP image.

// Create an empty Texture2D.
Texture2D texture = new Texture2D(0, 0);

// Load a file to a byte array.
string filename = "path/to/image";
byte[] rawData = System.IO.File.ReadAllBytes(filename);

// Load an byte array to the Texture2D.
texture.LoadImage(rawData);

Video file

A video may by an MP4, AVI, MOV, WEBM, or WMV file.

[SerializeField] VideoPlayer _videoPlayer;

// Create a RenderTexture with Videoplayer frame data.
RenderTexture renderTexture = (RenderTexture)_videoPlayer.texture;
RenderTexture.active = renderTexture;

// Load image from RenderTexture to a Texture2D.
Texture2D texture = new Texture2D(0, 0);
texture.ReadPixels(new Rect(0, 0, renderTexture.width, renderTexture.height), 0, 0);
texture.Apply();

Computer webcams, external USB cameras, and phone rear/selfie cameras are supported.

[SerializeField] private ImageView _image;
[SerializeField] private WebcamSource _webcam;

// Load the new frame to an ImageView.
_image.Load(_webcam);

// Load image from ImageView to a Texture2D.
Texture2D texture = _image.Texture;

Step 2: Detect hands

Pass the Texture2D from step 1 to a HandTracker. The HandTracker will detect the hands and fingers in the texture.

// Create a handTracker.
HandTracker tracker = new HandTracker();

// Detect hands.
List<Hand> hands = tracker.Load(texture);

Step 3: Access hand & finger data

Access the detected hand's information.

if (hands.Count > 0)
{
    // Get hand Side. 
    HandSide side = hands[0].Side;
    
    // Get hand Confidence. 
    float confidence = hands[0].Confidence;
    
    // Get Thumb finger.
    Thumb thumb = hands[0].Thumb;
    
    // Get Index finger.
    Index index = hands[0].Index;
    
    // Get Middle finger.
    Middle middle = hands[0].Middle;
    
    // Get Ring finger.
    Ring ring = hands[0].Ring;
    
    // Get Pinky finger.
    Pinky pinky = hands[0].Pinky;
    
    // Get Palm.
    Palm palm = hands[0].Palm;
    
    // Get Pinky MCP finger joint.
    FingerJoint pinkyMCP = pinky.MCP;
    
    // Get Pinky MCP joint 2D position.
    Vector2 screenPosition = pinkyMCP.Position2D;
    
    // Get Pinky MCP joint 3D position.
    Vector3 worldPosition = pinkyMCP.Position3D;
}

Examples

For more detailed examples, check our dedicated Examples section:

Camera feed

📷
Detect hand and retrieve joints
No Code: Detect and visualize hands from a Sprite
Detect and visualize hands from a Sprite
Detect and visualize hands from a video
Use webcam to detect and visualize hands (2D Canvas)
Use webcam to detect and visualize hands (3D World space)
🖼️
🎥
Workflow