# Use webcam to detect and visualize hands (3D World space) This example demonstrates how to load and display camera feed in a Unity scene with a [WebcamSource](https://handtracking.lightbuzz.com/ui-reference/streamsource/webcamsource) and an [ImageView](https://handtracking.lightbuzz.com/ui-reference/imageview), implement hand tracking with the [HandTracker](https://handtracking.lightbuzz.com/api-reference/handtracker), and use the [HandManager](https://handtracking.lightbuzz.com/ui-reference/handmanager) to render detected fingers in 3D world space. {% hint style="info" %} This is a code walkthrough of the `LightBuzz_Hand_Tracking_3D` Hand Tracking Unity plugin sample. The plugin includes the no-code demo that has the same functionality. {% endhint %} #### Step 1: Create a Unity scene Open the Unity Project you created in the [Installation](https://handtracking.lightbuzz.com/hand-tracking-unity-plugin/installation) section. Right-click on the `Assets` folder and select `Create > Scene`.

Type the scene's name. In this example, we'll use the name `WebcamDemo3D`. After the scene is created, right-click on the scene and select `GameObject > UI > Canvas`.

Navigate to the LightBuzz `Prefabs` folder at `Assets\LightBuzz Hand Tracking\Runtime\Prefabs`.

For this example, you'll need the [ImageView](https://handtracking.lightbuzz.com/ui-reference/imageview), [WebcamSource](https://handtracking.lightbuzz.com/ui-reference/streamsource/webcamsource) and [HandManager](https://handtracking.lightbuzz.com/ui-reference/handmanager) prefabs. Drag and drop the [ImageView](https://handtracking.lightbuzz.com/ui-reference/imageview) and [WebcamSource](https://handtracking.lightbuzz.com/ui-reference/streamsource/webcamsource) prefabs into the Canvas.

Drag and drop the [HandManager](https://handtracking.lightbuzz.com/ui-reference/handmanager) prefab on the `Hierarchy` pane (but not into the Canvas).

{% hint style="info" %} To see the 3D detections, the [HandManager](https://handtracking.lightbuzz.com/ui-reference/handmanager) needs to be outside of the Canvas. {% endhint %} Then, right-click on the `Hierarchy` pane and select `Create Empty`.

Give a name to the new component. In this example, we'll use the name `Demo`. Then, go to the `Inspector` pane and select `Add Component`. In the search bar, type `new` and select the `New script` option.

Type the script's name and select `Create and Add`. For this example, we'll use the name `WebcamDemo3D`.

#### Step 2: Initialize the visual components Double-click on the newly created `MonoBehaviour` script and import the necessary namespaces. ```csharp using LightBuzz.HandTracking; using System.Collections.Generic; using UnityEngine; ``` For this example, we'll need a [WebcamSource](https://handtracking.lightbuzz.com/ui-reference/streamsource/webcamsource) to get the frames, an [ImageView](https://handtracking.lightbuzz.com/ui-reference/imageview) to draw camera texture and a [HandManager](https://handtracking.lightbuzz.com/ui-reference/handmanager) to visualize the detected hands. ```csharp [SerializeField] private ImageView _image; [SerializeField] private WebcamSource _webcam; [SerializeField] private HandManager _handManager; ``` After adding the serialized fields, go to the Unity Editor to connect these fields with the `Demo` component. At the `Inspector` pane, select the round button next to each `SerializeField`.

Then, at the `Scene` tab, select the corresponding prefab. For example, for the `Image` field, select the `ImageView` prefab.

When all fields are connected, the result should resemble the following image.

Then, select the `HandManager` prefab. Connect the `Image` field to the `ImageView` prefab and uncheck the `Is 2D` option.

{% hint style="warning" %} Make sure the `Is 2D` option is NOT selected to see the hand tracking detections in the 3D world space. If the option is checked, detections are displayed in the 2D space. {% endhint %} After connecting all the fields, navigate to the `Canvas` to set the render options. Change the `Render Mode` to `Screen Space - Camera`.

Then, set the `Main Camera,` from the Scene tab, as the `Render Camera`.

When all the render options are set, the result should look like the following image.

#### Step 3: Adjust Scene components To display the hand detections in the 3D world space we need to adjust the [ImageView](https://handtracking.lightbuzz.com/ui-reference/imageview). Navigate to the [ImageView](https://handtracking.lightbuzz.com/ui-reference/imageview) prefab, under the `Canvas`, go to the `Aspect Ratio Fitter` section and change `Aspect Mode` to `None`.

Then, go to the `Rect Transform` section of the [ImageView](https://handtracking.lightbuzz.com/ui-reference/imageview) prefab and click on the blue cross to open the `Anchor Presets`.

Select the `middle` and `center` option (the one that looks like a target). Then, change the `Width` to `400`, the `Height` to `200` and the `Pos X` to `-600`.

{% hint style="info" %} The suggested `Rect Transform` settings are designed to reduce the size of the [ImageView](https://handtracking.lightbuzz.com/ui-reference/imageview) and reposition it to the side. Feel free to experiment and set the settings to different values. {% endhint %} For a proper view, navigate to the `Main Camera` component and set the `Y Position` to `0`, since the 3D coordinates of the hands are estimated relative to the cartesian origin point. Also, set the `Z Position` to `-1` to move the camera farther from the hands.

Finally, return to the `MonoBehaviour` script and instantiate a [HandTracker](https://handtracking.lightbuzz.com/api-reference/handtracker) to detect hands. ```csharp HandTracker _handTracker = new HandTracker(); ``` #### Step 4: Open the webcam Open the webcam to get the live feed. ```csharp _webcam.Open(); ``` {% hint style="info" %} In this example, the camera is opened in the `Start()` method. Alternatively, you could open the camera with the click of a button. {% endhint %} #### Step 5: Get the live feed Check that the camera is open and available for capturing video. ```csharp if (!_webcam.IsOpen) return; ``` Load the new frame from the `_webcam` object onto the [ImageView](https://handtracking.lightbuzz.com/ui-reference/imageview) to show the live feed to your users. ```csharp _image.Load(_webcam); ``` #### Step 6: Detect hands Pass the `Texture2D` object from the camera frame to the [HandTracker](https://handtracking.lightbuzz.com/api-reference/handtracker) for processing. ```csharp List hands = _handTracker.Load(_image.Texture); ``` The [HandTracker](https://handtracking.lightbuzz.com/api-reference/handtracker) will analyze the texture and detect any hands present in the image. #### Step 7: Visualize the hands To display the detected hands in 3D world space, first sort the hands by position X. ```csharp hands.Sort((h1, h2) => h1[FingerJointType.Root].Position2D.x.CompareTo( h2[FingerJointType.Root].Position2D.x)); ``` {% hint style="warning" %} By default, the [HandManager](https://handtracking.lightbuzz.com/ui-reference/handmanager) positions the root joint of any detected hand at the origin point (0, 0, 0) within the 3D world space. To track multiple hands and visualize them clearly within the 3D environment, you need to provide an offset value in meters for each hand. Provided with an offset list, the [HandManager](https://handtracking.lightbuzz.com/ui-reference/handmanager) will reposition each hand according to the offset value. {% endhint %} In this example, each new hand will be positioned 25 cm away from the previous one. ```csharp List offsets = new List(); float offset_x = 0f; float step = 0.25f; foreach (Hand hand in hands) { offsets.Add(new Vector3(offset_x, 0f, 0f)); offset_x += step; } ``` Then, simply pass the hand detections and the offsets to the [HandManager](https://handtracking.lightbuzz.com/ui-reference/handmanager). It will manage the rendering and updates required to accurately depict the hands in 3D world space based on the detection data provided. ```csharp _handManager.Load(hands, offsets); ``` {% hint style="info" %} Steps 5 through 7 are incorporated into the `Update()` method. {% endhint %} #### Step 8: Release the resources Close the webcam to stop the live feed, preventing further video capture. ```csharp _webcam.Close(); ``` Dispose of the [HandTracker](https://handtracking.lightbuzz.com/api-reference/handtracker) object to ensure that all associated resources are released. ```csharp _handTracker.Dispose(); ``` {% hint style="info" %} In this example, the resources are released in the `OnDestroy()` method. Alternatively, you could do that with the click of a button or in the `OnApplicationQuit()` method. {% endhint %} ### Full example code Here is the full example code that has the same functionality as the Hand Tracking Unity plugin `LightBuzz_Hand_Tracking_3D` sample.

using LightBuzz.HandTracking;
using System.Collections.Generic;
using UnityEngine;

public class WebcamDemo3D : MonoBehaviour
{
    [SerializeField] private ImageView _image;
    [SerializeField] private WebcamSource _webcam;
    [SerializeField] private HandManager _handManager;
    
    private readonly HandTracker _handTracker = new HandTracker();

    private void Start()
    {
        _webcam.Open();
    }

    private void OnDestroy()
    {
        _webcam.Close();
        _handTracker.Dispose();
    }

    private void Update()
    {
        if (!_webcam.IsOpen) return;

        // 1. Draw the camera texture.
        _image.Load(_webcam);

        // 2. Detect hands in the camera texture.
        List<Hand> hands = _handTracker.Load(_image.Texture);
        
        // 3. Sort hands by position X.
        hands.Sort((h1, h2) => 
            h1[FingerJointType.Root].Position2D.x.CompareTo(
                h2[FingerJointType.Root].Position2D.x));

        // 4. Add offset to show all detected hands.
        List<Vector3> offsets = new List<Vector3>();
        float offset_x = 0f;
        float step = 0.25f;

        foreach (Hand hand in hands)
        {
            offsets.Add(new Vector3(offset_x, 0f, 0f));
            offset_x += step;
        }

        // 5. Visualize the hands on a Canvas.
        _handManager.Load(hands, offsets);
    }
}

By following these steps, you will be able to load the camera feed into your application, detect hands in real time, and finally, render these detections in 3D world space.