> For the complete documentation index, see [llms.txt](https://handtracking.lightbuzz.com/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://handtracking.lightbuzz.com/examples/detect-and-visualize-hands-from-a-video.md).

# Detect and visualize hands from a video

This example demonstrates how to load and display a video in a Unity scene with a [VideoSource](/ui-reference/streamsource/videosource.md) and an [ImageView](/ui-reference/imageview.md), implement hand tracking with the [HandTracker](/api-reference/handtracker.md), and use the [HandManager](/ui-reference/handmanager.md) to render detected fingers on a 2D canvas.

{% hint style="info" %}
This is a code walkthrough of the `LightBuzz_Hand_Tracking_Video` Hand Tracking Unity plugin sample. The plugin includes the no-code demo that has the same functionality.
{% endhint %}

#### Step 1: Create a Unity scene

Open the Unity Project you created in the [Installation](/hand-tracking-unity-plugin/installation.md) section.

Right-click on the `Assets` folder and select `Create > Scene`.

<figure><img src="/files/AWAZBZqNkgh8mdgwr3hv" alt=""><figcaption><p>Create new scene</p></figcaption></figure>

Type the scene's name. In this example, we'll use the name `VideoDemo`.

After the scene is created, right-click on the scene and select `GameObject > UI > Canvas`.

<figure><img src="/files/SOVl2RObo6YAMFPnsd5J" alt=""><figcaption><p>Add a Canvas</p></figcaption></figure>

Navigate to the LightBuzz `Prefabs` folder at `Assets\LightBuzz Hand Tracking\Runtime\Prefabs`.

<figure><img src="/files/aLOxlnZxIBjwQFtfkzwZ" alt=""><figcaption><p>LightBuzz Prefabs</p></figcaption></figure>

For this example, drag and drop the [ImageView](/ui-reference/imageview.md), [VideoSource](/ui-reference/streamsource/videosource.md) and [HandManager](/ui-reference/handmanager.md) prefabs into the Canvas.

<figure><img src="/files/w1uyYtGbyf6lRt6pkKfE" alt=""><figcaption><p>Add prefabs</p></figcaption></figure>

Then, right-click on the `Hierarchy` pane and select `Create Empty`.&#x20;

<figure><img src="/files/tWKAoCSDw2MNbLtdn4Dg" alt=""><figcaption><p>Create empty component</p></figcaption></figure>

Give a name to the new component. In this example, we'll use the name `Demo`.

Then, go to the `Inspector` pane and select `Add Component`. In the search bar, type `new` and select the `New script` option.

<figure><img src="/files/aEWEKS46jXoNR0dQLnvd" alt=""><figcaption><p>Add script</p></figcaption></figure>

Type the script's name and select `Create and Add`. For this example, we'll use the name `VideoDemo`.

<figure><img src="/files/zJM0toCAYIptsc9LZEBp" alt=""><figcaption><p>Create script</p></figcaption></figure>

#### Step 2: Initialize the visual components

Double-click on the newly created `MonoBehaviour` script and import the necessary namespaces.

```csharp
using LightBuzz.HandTracking;
using System.Collections.Generic;
using UnityEngine;
using UnityEngine.Video;
```

For this example, we'll need a `VideoPlayer` to get the video frames, an [ImageView](/ui-reference/imageview.md) to draw image texture and a [HandManager](/ui-reference/handmanager.md) to visualize the detected hands.

```csharp
[SerializeField] private ImageView _image;
[SerializeField] private HandManager _handManager;
[SerializeField] VideoPlayer _videoPlayer;
```

Add a `Texture2D` object to load the video frames.

```csharp
private Texture2D _texture;
```

After adding the serialized fields, go to the Unity Editor to connect these fields with the `Demo` component.

At the `Inspector` pane, select the round button next to each `SerializeField`.

<figure><img src="/files/lt4X63LB362hxQnNwyPU" alt=""><figcaption><p>Seriaze fileds</p></figcaption></figure>

Then, at the `Scene` tab, select the corresponding prefab. For example, for the `Image` field, select the `ImageView` prefab.

<figure><img src="/files/0jqp9uI7zQIPnTTMXJIa" alt=""><figcaption><p>Connect prefab</p></figcaption></figure>

When all fields are connected, the result should resemble the following image.

<figure><img src="/files/HQdcZHi3nuYq4wblTYVI" alt=""><figcaption><p>Connected serialize fields</p></figcaption></figure>

Then, select the `HandManager` prefab, under the `Canvas`, and connect the `Image` field to the `ImageView` prefab.

<figure><img src="/files/J7W7y81Uo00EVQAPLrLA" alt=""><figcaption><p>Connect field to HandManager</p></figcaption></figure>

{% hint style="info" %}
Make sure the `Is 2D` option is selected to see the hand tracking detections in the 2D space. If the option is not checked, detections are displayed in the 3D world space.
{% endhint %}

Then, select the `VideoSource` prefab, under the `Canvas,` and connect the `Video Clip` field to a video source. You can either use the demo video `lightbuzz-hand-tracking.mp4` included in the `Assets/LightBuzz Hand Tracking/Runtime/Media` or create your own video source.

<figure><img src="/files/EPZhswY0XITRnC3ScNNC" alt=""><figcaption></figcaption></figure>

After connecting all the fields, navigate to the `Canvas` to set the render options.

Change the `Render Mode` to `Screen Space - Camera`.

<figure><img src="/files/8xapwQLfkO58oNi2gJhH" alt=""><figcaption><p>Change Render Mode</p></figcaption></figure>

Then, set the `Main Camera,` from the Scene tab, as the `Render Camera`.

<figure><img src="/files/FKcxZ5Eyq9JlM0GAFHZu" alt=""><figcaption><p>Select Render Camera</p></figcaption></figure>

When all the render options are set, the result should look like the following image.

<figure><img src="/files/bVxgPsw5dk4FXW0xcKXy" alt=""><figcaption><p>Render settings</p></figcaption></figure>

Finally, return to the script and instantiate a [HandTracker](/api-reference/handtracker.md) to detect hands.

```csharp
HandTracker _handTracker = new HandTracker();
```

#### Step 3: Register handler methods to VideoPlayer events

Enable the `frameReady` events. Any registered delegates with `VideoPlayer.frameReady` will be invoked when a frame is ready to be drawn.

```csharp
 _videoPlayer.sendFrameReadyEvents = true;
```

Register for the `VideoPlayer.prepareCompleted` event with the `VideoPrepared` event handler method.

<pre class="language-csharp"><code class="lang-csharp"><strong>_videoPlayer.prepareCompleted += VideoPrepared;
</strong></code></pre>

Register for the `VideoPlayer.frameReady` event with the `VideoFrameReady` event handler method.&#x20;

```csharp
_videoPlayer.frameReady += VideoFrameReady;
```

Initiate playback engine preparation. After this is done, video frames can be received immediately.

<pre class="language-csharp"><code class="lang-csharp"><strong>_videoPlayer.Prepare();
</strong></code></pre>

{% hint style="info" %}
In this example, the `VideoPlayer` event handler methods are registered in the `Start()` method.
{% endhint %}

#### Step 4: Implement the VideoPrepared handler

Create the `Texture2D` according to the `VideoPlayer` resolution.

```
_texture = new Texture2D
    (
        (int)source.width,
        (int)source.height,
        TextureFormat.RGB24,
        false
    );
```

Start playback.

```csharp
_videoPlayer.Play();
```

{% hint style="info" %}
In this example, the prepare and playback functionality is implemented in the `VideoPrepared` event handler method.
{% endhint %}

#### Step 5: Implement the VideoFrameReadyhandler

Create a `RenderTexture` with the `VideoPlayer` frame data.

```csharp
RenderTexture renderTexture = (RenderTexture)_videoPlayer.texture;
RenderTexture.active = renderTexture;

_texture.ReadPixels(new Rect(0, 0, renderTexture.width, renderTexture.height), 0, 0);
_texture.Apply();
```

Load the texture onto the [ImageView](/ui-reference/imageview.md) to show the video feed.

```csharp
_image.Load(_texture);
```

Pass the `texture` to the [HandTracker](/api-reference/handtracker.md) for processing.

```csharp
List<Hand> hands = _handTracker.Load(_texture);
```

The [HandTracker](/api-reference/handtracker.md) will analyze the texture and detect any hands present in the image.

Pass the detections to the [HandManager](/ui-reference/handmanager.md).  The [HandManager](/ui-reference/handmanager.md) will manage the rendering and updates required to depict the detected hands.

```csharp
_handManager.Load(hands);
```

{% hint style="info" %}
In this example, all the hand-tracking functionality is implemented in the `VideoFrameReady` event handler method.
{% endhint %}

#### Step 6: Release the resources

Unsubscribe the `VideoPrepared` event handler method from the `VideoPlayer.prepareCompleted` event.

```csharp
_videoPlayer.prepareCompleted -= VideoPrepared;
```

Unsubscribe the `VideoFrameReady` event handler method from the `VideoPlayer.frameReady` event.

```csharp
_videoPlayer.frameReady -= VideoFrameReady;
```

Dispose of the [HandTracker](/api-reference/handtracker.md) object to ensure that all associated resources are released.

```csharp
_handTracker.Dispose();
```

Destroy the `Texture2D` object created in Step 2.

```csharp
Destroy(_texture);
```

{% hint style="info" %}
In this example, the resources are released in the `OnDestroy()` method. Alternatively, you could do that in the `OnApplicationQuit()` method.
{% endhint %}

### Full example code

Here is the full example code that has the same functionality as the Hand Tracking Unity plugin `LightBuzz_Hand_Tracking_Video` sample.

```csharp
using LightBuzz.HandTracking;
using System;
using System.Collections.Generic;
using UnityEngine;
using UnityEngine.Video;

public class VideoDemo : MonoBehaviour
{
    [SerializeField] private ImageView _image;
    [SerializeField] private HandManager _handManager;
    [SerializeField] VideoPlayer _videoPlayer;

    private Texture2D _texture;

    private readonly HandTracker _handTracker = new HandTracker();

    private void Start()
    {
        // Enable the frameReady events.
        _videoPlayer.sendFrameReadyEvents = true;
    
        // Register the VideoPrepared handler method.
        _videoPlayer.prepareCompleted += VideoPrepared;
    
        // Register the VideoFrameReady handler method.
        _videoPlayer.frameReady += VideoFrameReady;
    
        // Initiate playback engine preparation.
        _videoPlayer.Prepare();
    }

    private void OnDestroy()
    {
        // Remove the VideoPrepared handler method.
        _videoPlayer.prepareCompleted -= VideoPrepared;
      
        // Unsubscribe the VideoFrameReady handler method.
        _videoPlayer.frameReady -= VideoFrameReady;
      
        // Dispose of the HandTracker.
        _handTracker.Dispose();
      
        // Destroy the texture.
        Destroy(_texture);
    }

    private void VideoPrepared(VideoPlayer source)
    {
        // Assign the VideoPlayer data to texture.
        _texture = new Texture2D((int)source.width, (int)source.height, TextureFormat.RGB24, false);
    
        // Start playback. 
        _videoPlayer.Play();
    }

    private void VideoFrameReady(VideoPlayer source, long frameIdx)
    {
        // Create a RenderTexture with Videoplayer frame data.
        RenderTexture renderTexture = (RenderTexture)_videoPlayer.texture;
        RenderTexture.active = renderTexture;

        _texture.ReadPixels(new Rect(0, 0, renderTexture.width, renderTexture.height), 0, 0);
        _texture.Apply();

        // Draw the camera texture.
        _image.Load(_texture);

        // Detect hands in the texture.
        List<Hand> hands = _handTracker.Load(_texture);

        // Visualize the hands on a Canvas.
        _handManager.Load(hands);
    }
}
```

By following these steps, you will be able to load the video feed into your application, detect hands, and finally, render these detections on a 2D canvas.