# Detect and visualize hands from a video

This example demonstrates how to load and display a video in a Unity scene with a [VideoSource](/ui-reference/streamsource/videosource.md) and an [ImageView](/ui-reference/imageview.md), implement hand tracking with the [HandTracker](/api-reference/handtracker.md), and use the [HandManager](/ui-reference/handmanager.md) to render detected fingers on a 2D canvas.

{% hint style="info" %}
This is a code walkthrough of the `LightBuzz_Hand_Tracking_Video` Hand Tracking Unity plugin sample. The plugin includes the no-code demo that has the same functionality.
{% endhint %}

#### Step 1: Create a Unity scene

Open the Unity Project you created in the [Installation](/hand-tracking-unity-plugin/installation.md) section.

Right-click on the `Assets` folder and select `Create > Scene`.

<figure><img src="/files/AWAZBZqNkgh8mdgwr3hv" alt=""><figcaption><p>Create new scene</p></figcaption></figure>

Type the scene's name. In this example, we'll use the name `VideoDemo`.

After the scene is created, right-click on the scene and select `GameObject > UI > Canvas`.

<figure><img src="/files/SOVl2RObo6YAMFPnsd5J" alt=""><figcaption><p>Add a Canvas</p></figcaption></figure>

Navigate to the LightBuzz `Prefabs` folder at `Assets\LightBuzz Hand Tracking\Runtime\Prefabs`.

<figure><img src="/files/aLOxlnZxIBjwQFtfkzwZ" alt=""><figcaption><p>LightBuzz Prefabs</p></figcaption></figure>

For this example, drag and drop the [ImageView](/ui-reference/imageview.md), [VideoSource](/ui-reference/streamsource/videosource.md) and [HandManager](/ui-reference/handmanager.md) prefabs into the Canvas.

<figure><img src="/files/w1uyYtGbyf6lRt6pkKfE" alt=""><figcaption><p>Add prefabs</p></figcaption></figure>

Then, right-click on the `Hierarchy` pane and select `Create Empty`.&#x20;

<figure><img src="/files/tWKAoCSDw2MNbLtdn4Dg" alt=""><figcaption><p>Create empty component</p></figcaption></figure>

Give a name to the new component. In this example, we'll use the name `Demo`.

Then, go to the `Inspector` pane and select `Add Component`. In the search bar, type `new` and select the `New script` option.

<figure><img src="/files/aEWEKS46jXoNR0dQLnvd" alt=""><figcaption><p>Add script</p></figcaption></figure>

Type the script's name and select `Create and Add`. For this example, we'll use the name `VideoDemo`.

<figure><img src="/files/zJM0toCAYIptsc9LZEBp" alt=""><figcaption><p>Create script</p></figcaption></figure>

#### Step 2: Initialize the visual components

Double-click on the newly created `MonoBehaviour` script and import the necessary namespaces.

```csharp
using LightBuzz.HandTracking;
using System.Collections.Generic;
using UnityEngine;
using UnityEngine.Video;
```

For this example, we'll need a `VideoPlayer` to get the video frames, an [ImageView](/ui-reference/imageview.md) to draw image texture and a [HandManager](/ui-reference/handmanager.md) to visualize the detected hands.

```csharp
[SerializeField] private ImageView _image;
[SerializeField] private HandManager _handManager;
[SerializeField] VideoPlayer _videoPlayer;
```

Add a `Texture2D` object to load the video frames.

```csharp
private Texture2D _texture;
```

After adding the serialized fields, go to the Unity Editor to connect these fields with the `Demo` component.

At the `Inspector` pane, select the round button next to each `SerializeField`.

<figure><img src="/files/lt4X63LB362hxQnNwyPU" alt=""><figcaption><p>Seriaze fileds</p></figcaption></figure>

Then, at the `Scene` tab, select the corresponding prefab. For example, for the `Image` field, select the `ImageView` prefab.

<figure><img src="/files/0jqp9uI7zQIPnTTMXJIa" alt=""><figcaption><p>Connect prefab</p></figcaption></figure>

When all fields are connected, the result should resemble the following image.

<figure><img src="/files/HQdcZHi3nuYq4wblTYVI" alt=""><figcaption><p>Connected serialize fields</p></figcaption></figure>

Then, select the `HandManager` prefab, under the `Canvas`, and connect the `Image` field to the `ImageView` prefab.

<figure><img src="/files/J7W7y81Uo00EVQAPLrLA" alt=""><figcaption><p>Connect field to HandManager</p></figcaption></figure>

{% hint style="info" %}
Make sure the `Is 2D` option is selected to see the hand tracking detections in the 2D space. If the option is not checked, detections are displayed in the 3D world space.
{% endhint %}

Then, select the `VideoSource` prefab, under the `Canvas,` and connect the `Video Clip` field to a video source. You can either use the demo video `lightbuzz-hand-tracking.mp4` included in the `Assets/LightBuzz Hand Tracking/Runtime/Media` or create your own video source.

<figure><img src="/files/EPZhswY0XITRnC3ScNNC" alt=""><figcaption></figcaption></figure>

After connecting all the fields, navigate to the `Canvas` to set the render options.

Change the `Render Mode` to `Screen Space - Camera`.

<figure><img src="/files/8xapwQLfkO58oNi2gJhH" alt=""><figcaption><p>Change Render Mode</p></figcaption></figure>

Then, set the `Main Camera,` from the Scene tab, as the `Render Camera`.

<figure><img src="/files/FKcxZ5Eyq9JlM0GAFHZu" alt=""><figcaption><p>Select Render Camera</p></figcaption></figure>

When all the render options are set, the result should look like the following image.

<figure><img src="/files/bVxgPsw5dk4FXW0xcKXy" alt=""><figcaption><p>Render settings</p></figcaption></figure>

Finally, return to the script and instantiate a [HandTracker](/api-reference/handtracker.md) to detect hands.

```csharp
HandTracker _handTracker = new HandTracker();
```

#### Step 3: Register handler methods to VideoPlayer events

Enable the `frameReady` events. Any registered delegates with `VideoPlayer.frameReady` will be invoked when a frame is ready to be drawn.

```csharp
 _videoPlayer.sendFrameReadyEvents = true;
```

Register for the `VideoPlayer.prepareCompleted` event with the `VideoPrepared` event handler method.

<pre class="language-csharp"><code class="lang-csharp"><strong>_videoPlayer.prepareCompleted += VideoPrepared;
</strong></code></pre>

Register for the `VideoPlayer.frameReady` event with the `VideoFrameReady` event handler method.&#x20;

```csharp
_videoPlayer.frameReady += VideoFrameReady;
```

Initiate playback engine preparation. After this is done, video frames can be received immediately.

<pre class="language-csharp"><code class="lang-csharp"><strong>_videoPlayer.Prepare();
</strong></code></pre>

{% hint style="info" %}
In this example, the `VideoPlayer` event handler methods are registered in the `Start()` method.
{% endhint %}

#### Step 4: Implement the VideoPrepared handler

Create the `Texture2D` according to the `VideoPlayer` resolution.

```
_texture = new Texture2D
    (
        (int)source.width,
        (int)source.height,
        TextureFormat.RGB24,
        false
    );
```

Start playback.

```csharp
_videoPlayer.Play();
```

{% hint style="info" %}
In this example, the prepare and playback functionality is implemented in the `VideoPrepared` event handler method.
{% endhint %}

#### Step 5: Implement the VideoFrameReadyhandler

Create a `RenderTexture` with the `VideoPlayer` frame data.

```csharp
RenderTexture renderTexture = (RenderTexture)_videoPlayer.texture;
RenderTexture.active = renderTexture;

_texture.ReadPixels(new Rect(0, 0, renderTexture.width, renderTexture.height), 0, 0);
_texture.Apply();
```

Load the texture onto the [ImageView](/ui-reference/imageview.md) to show the video feed.

```csharp
_image.Load(_texture);
```

Pass the `texture` to the [HandTracker](/api-reference/handtracker.md) for processing.

```csharp
List<Hand> hands = _handTracker.Load(_texture);
```

The [HandTracker](/api-reference/handtracker.md) will analyze the texture and detect any hands present in the image.

Pass the detections to the [HandManager](/ui-reference/handmanager.md).  The [HandManager](/ui-reference/handmanager.md) will manage the rendering and updates required to depict the detected hands.

```csharp
_handManager.Load(hands);
```

{% hint style="info" %}
In this example, all the hand-tracking functionality is implemented in the `VideoFrameReady` event handler method.
{% endhint %}

#### Step 6: Release the resources

Unsubscribe the `VideoPrepared` event handler method from the `VideoPlayer.prepareCompleted` event.

```csharp
_videoPlayer.prepareCompleted -= VideoPrepared;
```

Unsubscribe the `VideoFrameReady` event handler method from the `VideoPlayer.frameReady` event.

```csharp
_videoPlayer.frameReady -= VideoFrameReady;
```

Dispose of the [HandTracker](/api-reference/handtracker.md) object to ensure that all associated resources are released.

```csharp
_handTracker.Dispose();
```

Destroy the `Texture2D` object created in Step 2.

```csharp
Destroy(_texture);
```

{% hint style="info" %}
In this example, the resources are released in the `OnDestroy()` method. Alternatively, you could do that in the `OnApplicationQuit()` method.
{% endhint %}

### Full example code

Here is the full example code that has the same functionality as the Hand Tracking Unity plugin `LightBuzz_Hand_Tracking_Video` sample.

```csharp
using LightBuzz.HandTracking;
using System;
using System.Collections.Generic;
using UnityEngine;
using UnityEngine.Video;

public class VideoDemo : MonoBehaviour
{
    [SerializeField] private ImageView _image;
    [SerializeField] private HandManager _handManager;
    [SerializeField] VideoPlayer _videoPlayer;

    private Texture2D _texture;

    private readonly HandTracker _handTracker = new HandTracker();

    private void Start()
    {
        // Enable the frameReady events.
        _videoPlayer.sendFrameReadyEvents = true;
    
        // Register the VideoPrepared handler method.
        _videoPlayer.prepareCompleted += VideoPrepared;
    
        // Register the VideoFrameReady handler method.
        _videoPlayer.frameReady += VideoFrameReady;
    
        // Initiate playback engine preparation.
        _videoPlayer.Prepare();
    }

    private void OnDestroy()
    {
        // Remove the VideoPrepared handler method.
        _videoPlayer.prepareCompleted -= VideoPrepared;
      
        // Unsubscribe the VideoFrameReady handler method.
        _videoPlayer.frameReady -= VideoFrameReady;
      
        // Dispose of the HandTracker.
        _handTracker.Dispose();
      
        // Destroy the texture.
        Destroy(_texture);
    }

    private void VideoPrepared(VideoPlayer source)
    {
        // Assign the VideoPlayer data to texture.
        _texture = new Texture2D((int)source.width, (int)source.height, TextureFormat.RGB24, false);
    
        // Start playback. 
        _videoPlayer.Play();
    }

    private void VideoFrameReady(VideoPlayer source, long frameIdx)
    {
        // Create a RenderTexture with Videoplayer frame data.
        RenderTexture renderTexture = (RenderTexture)_videoPlayer.texture;
        RenderTexture.active = renderTexture;

        _texture.ReadPixels(new Rect(0, 0, renderTexture.width, renderTexture.height), 0, 0);
        _texture.Apply();

        // Draw the camera texture.
        _image.Load(_texture);

        // Detect hands in the texture.
        List<Hand> hands = _handTracker.Load(_texture);

        // Visualize the hands on a Canvas.
        _handManager.Load(hands);
    }
}
```

By following these steps, you will be able to load the video feed into your application, detect hands, and finally, render these detections on a 2D canvas.


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://handtracking.lightbuzz.com/examples/detect-and-visualize-hands-from-a-video.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.