# Detect and visualize hands from a Sprite

This example demonstrates how to load and display image sprites in a Unity scene with an [ImageView](https://handtracking.lightbuzz.com/ui-reference/imageview), implement hand tracking with the [HandTracker](https://handtracking.lightbuzz.com/api-reference/handtracker), and use the [HandManager](https://handtracking.lightbuzz.com/ui-reference/handmanager) to render detected fingers on a 2D canvas.

{% hint style="info" %}
This is a code walkthrough of the `LightBuzz_Hand_Tracking_Picture` Hand Tracking Unity plugin sample. The plugin includes the no-code demo that has the same functionality.
{% endhint %}

#### Step 1: Create a Unity scene

Open the Unity Project you created in the [Installation](https://handtracking.lightbuzz.com/hand-tracking-unity-plugin/installation) section.

Right-click on the `Assets` folder and select `Create > Scene`.

<figure><img src="https://1895788644-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FyGqKFJQGBmrlsrsolzNU%2Fuploads%2Fjq6n6qCxjh0OR775jrfm%2F1_create_scene.png?alt=media&#x26;token=f8e0e814-512e-4bab-b17e-c341a8bd2281" alt=""><figcaption><p>Create new scene</p></figcaption></figure>

Type the scene's name. In this example, we'll use the name `PictureDemo`.

After the scene is created, right-click on the scene and select `GameObject > UI > Canvas`.

<figure><img src="https://1895788644-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FyGqKFJQGBmrlsrsolzNU%2Fuploads%2Fde9seJ1XF1in693SVhhX%2F2_add_canvas.png?alt=media&#x26;token=dc33f608-7150-4fe9-a2c4-07fa0e212c35" alt=""><figcaption><p>Add a Canvas</p></figcaption></figure>

Navigate to the LightBuzz `Prefabs` folder at `Assets\LightBuzz Hand Tracking\Runtime\Prefabs`.

<figure><img src="https://1895788644-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FyGqKFJQGBmrlsrsolzNU%2Fuploads%2FptBTcWOYXEUCIgv8n8Cs%2F3_prefabs.png?alt=media&#x26;token=a8c96402-2def-43f9-87cb-764c0336e3ed" alt=""><figcaption><p>LightBuzz Prefabs</p></figcaption></figure>

For this example, drag and drop the [ImageView](https://handtracking.lightbuzz.com/ui-reference/imageview) and [HandManager](https://handtracking.lightbuzz.com/ui-reference/handmanager) prefabs into the Canvas.

<figure><img src="https://1895788644-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FyGqKFJQGBmrlsrsolzNU%2Fuploads%2Fsk5RNM8NlaYUoxsI968x%2F4_add_prefabs_picture.png?alt=media&#x26;token=f575616d-3a92-41e2-acb8-32ddf2114b94" alt=""><figcaption><p>Add prefabs</p></figcaption></figure>

Then, right-click on the `Hierarchy` pane and select `Create Empty`.&#x20;

<figure><img src="https://1895788644-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FyGqKFJQGBmrlsrsolzNU%2Fuploads%2F128JJiRWg5tVPvbu5wAZ%2F5_create_empty.png?alt=media&#x26;token=1388ed19-e6ee-4b1d-b2cc-32d51aa6882e" alt=""><figcaption><p>Create empty component</p></figcaption></figure>

Give a name to the new component. In this example, we'll use the name `Demo`.

Then, go to the `Inspector` pane and select `Add Component`. In the search bar, type `new` and select the `New script` option.

<figure><img src="https://1895788644-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FyGqKFJQGBmrlsrsolzNU%2Fuploads%2FOmSwepgdDduiVmMQubl0%2F6_add_script.png?alt=media&#x26;token=627e78d1-8fd0-4408-933b-2cfe6f2fbb81" alt=""><figcaption><p>Add script</p></figcaption></figure>

Type the script's name and select `Create and Add`. For this example, we'll use the name `PictureDemo`.

<figure><img src="https://1895788644-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FyGqKFJQGBmrlsrsolzNU%2Fuploads%2Fp32lxe5x1b18lF1BbhDp%2F7_name_script_picture.png?alt=media&#x26;token=4e4db285-9544-4f6c-a304-798b75e0cf6d" alt=""><figcaption><p>Create script</p></figcaption></figure>

#### Step 2: Initialize the visual components

Double-click on the newly created `MonoBehaviour` script and import the necessary namespaces.

```csharp
using LightBuzz.HandTracking;
using System.Collections.Generic;
using UnityEngine;
```

For this example, we'll need a `Sprite`to load an image, an [ImageView](https://handtracking.lightbuzz.com/ui-reference/imageview) to draw its texture and a [HandManager](https://handtracking.lightbuzz.com/ui-reference/handmanager) to visualize the detected hands.

```csharp
[SerializeField] private ImageView _image;
[SerializeField] private HandManager _handManager;
[SerializeField] private Sprite _sprite;
```

After adding the serialized fields, go to the Unity Editor to connect these fields with the `Demo` component.

At the `Inspector` pane, select the round button next to each `SerializeField`.

<figure><img src="https://1895788644-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FyGqKFJQGBmrlsrsolzNU%2Fuploads%2FdS49z7mMVJej335XJtIz%2F8_connect_serialize_fields.png?alt=media&#x26;token=6f39c5d1-262b-4b17-9274-150cca89b2ba" alt=""><figcaption><p>Seriaze fileds</p></figcaption></figure>

Then, at the `Scene` tab, select the corresponding prefab. For example, for the `Image` field, select the `ImageView` prefab.

<figure><img src="https://1895788644-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FyGqKFJQGBmrlsrsolzNU%2Fuploads%2Fu5hbIW0JSuqVkmPJJ04O%2F9_imageview.png?alt=media&#x26;token=87365163-98eb-424a-af4b-859ca882f91c" alt=""><figcaption><p>Connect prefab</p></figcaption></figure>

The `Sprite` field should connect to an image sprite. You can either use the demo image sprite `lightbuzz-hand-tracking.png` included in the `Assets/LightBuzz Hand Tracking/Runtime/Media` folder or create your own image sprite.

When all fields are connected, the result should resemble the following image.

<figure><img src="https://1895788644-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FyGqKFJQGBmrlsrsolzNU%2Fuploads%2FZbuBZYpUDB6Z5gni9woO%2F10_connected_prefabs.png?alt=media&#x26;token=1c94ef74-9855-46d1-a8d5-95b11dba5e20" alt=""><figcaption><p>Connected serialize fields</p></figcaption></figure>

Then, select the `HandManager` prefab, under the `Canvas`, and connect the `Image` field to the `ImageView` prefab.

<figure><img src="https://1895788644-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FyGqKFJQGBmrlsrsolzNU%2Fuploads%2FlKAsHvTsDJeE9mM2KW73%2F10a_connected_handmanager.png?alt=media&#x26;token=02226392-8b7b-4fa3-a01b-0848744f75dd" alt=""><figcaption><p>Connect field to HandManager</p></figcaption></figure>

{% hint style="info" %}
Make sure the `Is 2D` option is selected to see the hand tracking detections in the 2D space. If the option is not checked, detections are displayed in the 3D world space.
{% endhint %}

After connecting all the fields, navigate to the `Canvas` to set the render options.

Change the `Render Mode` to `Screen Space - Camera`.

<figure><img src="https://1895788644-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FyGqKFJQGBmrlsrsolzNU%2Fuploads%2F3Nl9z2Beuk8eRI1DZj7G%2F11_render_mode_camera.png?alt=media&#x26;token=e582de34-c7cb-4610-82a7-9d2a3b9f486d" alt=""><figcaption><p>Change Render Mode</p></figcaption></figure>

Then, set the `Main Camera,` from the Scene tab, as the `Render Camera`.

<figure><img src="https://1895788644-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FyGqKFJQGBmrlsrsolzNU%2Fuploads%2FIUjjpjWLwZqDxqRdJ5q9%2F12_main_camera.png?alt=media&#x26;token=d82b6dce-3e08-4777-8bb2-23ed475d6aca" alt=""><figcaption><p>Select Render Camera</p></figcaption></figure>

When all the render options are set, the result should look like the following image.

<figure><img src="https://1895788644-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FyGqKFJQGBmrlsrsolzNU%2Fuploads%2FqVosuFDLymh0LRB1qK48%2F13_render_camera_settings.png?alt=media&#x26;token=6a83eba6-4c2c-4787-a324-f1c519b26026" alt=""><figcaption><p>Render settings</p></figcaption></figure>

Finally, return to the script and instantiate a [HandTracker](https://handtracking.lightbuzz.com/api-reference/handtracker) to detect hands.

```csharp
HandTracker _handTracker = new HandTracker();
```

#### Step 3: Load the image

Create a `Texture2D` object with the `Sprite`'s texture.

```csharp
Texture2D texture = _sprite.texture;
```

Load the `Texture2D` object onto the [ImageView](https://handtracking.lightbuzz.com/ui-reference/imageview) to show the image.

```csharp
_image.Load(texture);
```

#### Step 4: Detect hands

Pass the texture created in Step 3 to the [HandTracker](https://handtracking.lightbuzz.com/api-reference/handtracker) for processing.

```csharp
List<Hand> hands = _handTracker.Load(_image.Texture);
```

The [HandTracker](https://handtracking.lightbuzz.com/api-reference/handtracker) will analyze the texture and detect any hands present in the image.

#### Step 5: Visualize the hands

To display the detected hands on a 2D canvas, simply pass the detections to the [HandManager](https://handtracking.lightbuzz.com/ui-reference/handmanager).  It will manage the rendering and updates required to accurately depict the hands on the canvas based on the detection data provided.

```csharp
_handManager.Load(hands);
```

{% hint style="info" %}
Steps 3 through 5 are incorporated into the `Start()` method.
{% endhint %}

#### Step 6: Release the resources

Dispose of the [HandTracker](https://handtracking.lightbuzz.com/api-reference/handtracker) object to ensure that all associated resources are released.

```csharp
_handTracker.Dispose();
```

{% hint style="info" %}
In this example, the resources are released in the `OnDestroy()` method. Alternatively, you could do that with the click of a button or in the `OnApplicationQuit()` method.
{% endhint %}

### Full example code

Here is the full example code that has the same functionality as the Hand Tracking Unity plugin  `LightBuzz_Hand_Tracking_Picture` sample.

```csharp
using LightBuzz.HandTracking;
using System.Collections.Generic;
using UnityEngine;

public class PictureDemo : MonoBehaviour
{
    [SerializeField] private ImageView _image;
    [SerializeField] private HandManager _handManager;
    [SerializeField] private Sprite _sprite;

    private readonly HandTracker _handTracker = new HandTracker();

    private void Start()
    {
        Texture2D texture = _sprite.texture;

        _image.Load(texture);

        List<Hand> hands = _handTracker.Load(_image.Texture);
        
        _handManager.Load(hands);
    }

    private void OnDestroy()
    {
        _handTracker.Dispose();
    }
}
```

By following these steps, you will be able to load a `Sprite` into your application, detect hands, and finally, render these detections on a 2D canvas.
