A QuickStart overview on how to detect hands and retrieve finger joint information.
Following the steps described in the Workflow section, here is how to detect hands and fingers in 3 easy steps:
Step 1: Load image data
Load an image to a Texture2D. The image could be loaded from one of the following sources:
🖼️ Image file
An image file is a PNG, JPG, WEBP, or BMP image.
// Create an empty Texture2D.
Texture2D texture = new Texture2D(0, 0);
// Load a file to a byte array.
string filename = "path/to/image";
byte[] rawData = System.IO.File.ReadAllBytes(filename);
// Load an byte array to the Texture2D.
texture.LoadImage(rawData);
🎥 Video file
A video may by an MP4, AVI, MOV, WEBM, or WMV file.
[SerializeField] VideoPlayer _videoPlayer;
// Create a RenderTexture with Videoplayer frame data.
RenderTexture renderTexture = (RenderTexture)_videoPlayer.texture;
RenderTexture.active = renderTexture;
// Load image from RenderTexture to a Texture2D.
Texture2D texture = new Texture2D(0, 0);
texture.ReadPixels(new Rect(0, 0, renderTexture.width, renderTexture.height), 0, 0);
texture.Apply();
📷 Camera feed
Computer webcams, external USB cameras, and phone rear/selfie cameras are supported.
[SerializeField] private ImageView _image;
[SerializeField] private WebcamSource _webcam;
// Load the new frame to an ImageView.
_image.Load(_webcam);
// Load image from ImageView to a Texture2D.
Texture2D texture = _image.Texture;
Step 2: Detect hands
Pass the Texture2D from step 1 to a HandTracker. The HandTracker will detect the hands and fingers in the texture.
// Create a handTracker.
HandTracker tracker = new HandTracker();
// Detect hands.
List<Hand> hands = tracker.Load(texture);
Step 3: Access hand & finger data
Access the detected hand's information.
if (hands.Count > 0)
{
// Get hand Side.
HandSide side = hands[0].Side;
// Get hand Confidence.
float confidence = hands[0].Confidence;
// Get Thumb finger.
Thumb thumb = hands[0].Thumb;
// Get Index finger.
Index index = hands[0].Index;
// Get Middle finger.
Middle middle = hands[0].Middle;
// Get Ring finger.
Ring ring = hands[0].Ring;
// Get Pinky finger.
Pinky pinky = hands[0].Pinky;
// Get Palm.
Palm palm = hands[0].Palm;
// Get Pinky MCP finger joint.
FingerJoint pinkyMCP = pinky.MCP;
// Get Pinky MCP joint 2D position.
Vector2 screenPosition = pinkyMCP.Position2D;
// Get Pinky MCP joint 3D position.
Vector3 worldPosition = pinkyMCP.Position3D;
}
Examples
For more detailed examples, check our dedicated Examples section: