QuickStart

A QuickStart overview on how to detect hands and retrieve finger joint information.

Following the steps described in the Workflow section, here is how to detect hands and fingers in 3 easy steps:

Step 1: Load image data

Load an image to a Texture2D. The image could be loaded from one of the following sources:

🖼️ Image file

An image file is a PNG, JPG, WEBP, or BMP image.

// Create an empty Texture2D.
Texture2D texture = new Texture2D(0, 0);

// Load a file to a byte array.
string filename = "path/to/image";
byte[] rawData = System.IO.File.ReadAllBytes(filename);

// Load an byte array to the Texture2D.
texture.LoadImage(rawData);

🎥 Video file

A video may by an MP4, AVI, MOV, WEBM, or WMV file.

[SerializeField] VideoPlayer _videoPlayer;

// Create a RenderTexture with Videoplayer frame data.
RenderTexture renderTexture = (RenderTexture)_videoPlayer.texture;
RenderTexture.active = renderTexture;

// Load image from RenderTexture to a Texture2D.
Texture2D texture = new Texture2D(0, 0);
texture.ReadPixels(new Rect(0, 0, renderTexture.width, renderTexture.height), 0, 0);
texture.Apply();

📷 Camera feed

Computer webcams, external USB cameras, and phone rear/selfie cameras are supported.

[SerializeField] private ImageView _image;
[SerializeField] private WebcamSource _webcam;

// Load the new frame to an ImageView.
_image.Load(_webcam);

// Load image from ImageView to a Texture2D.
Texture2D texture = _image.Texture;

Step 2: Detect hands

Pass the Texture2D from step 1 to a HandTracker. The HandTracker will detect the hands and fingers in the texture.

// Create a handTracker.
HandTracker tracker = new HandTracker();

// Detect hands.
List<Hand> hands = tracker.Load(texture);

Step 3: Access hand & finger data

Access the detected hand's information.

if (hands.Count > 0)
{
    // Get hand Side. 
    HandSide side = hands[0].Side;
    
    // Get hand Confidence. 
    float confidence = hands[0].Confidence;
    
    // Get Thumb finger.
    Thumb thumb = hands[0].Thumb;
    
    // Get Index finger.
    Index index = hands[0].Index;
    
    // Get Middle finger.
    Middle middle = hands[0].Middle;
    
    // Get Ring finger.
    Ring ring = hands[0].Ring;
    
    // Get Pinky finger.
    Pinky pinky = hands[0].Pinky;
    
    // Get Palm.
    Palm palm = hands[0].Palm;
    
    // Get Pinky MCP finger joint.
    FingerJoint pinkyMCP = pinky.MCP;
    
    // Get Pinky MCP joint 2D position.
    Vector2 screenPosition = pinkyMCP.Position2D;
    
    // Get Pinky MCP joint 3D position.
    Vector3 worldPosition = pinkyMCP.Position3D;
}

Examples

For more detailed examples, check our dedicated Examples section:

Detect hand and retrieve jointsNo Code: Detect and visualize hands from a SpriteDetect and visualize hands from a SpriteDetect and visualize hands from a videoUse webcam to detect and visualize hands (2D Canvas)Use webcam to detect and visualize hands (3D World space)