Vuer

Hands

The Hands component enables WebXR hand tracking for VR/AR applications. This is essential for:

  • Natural hand interaction in VR
  • Building gesture-based interfaces
  • Creating immersive teleoperation systems
  • Developing hand-tracking research applications

Basic Usage

A minimal example that enables hand tracking:

python
import asyncio
from vuer import Vuer, VuerSession
from vuer.schemas import Hands

app = Vuer()

@app.add_handler("HAND_MOVE")
async def handler(event, session):
    print(f"Hand Event: key-{event.key}", event.value)

@app.spawn(start=True)
async def main(session: VuerSession):
    session.upsert(
        Hands(
            stream=True,
            key="hands",
        ),
        to="bgChildren",
    )

    while True:
        await asyncio.sleep(1.0)

Key Parameters

ParameterTypeDefaultDescription
keystr-Unique identifier for the hands component
streamboolFalseEnable streaming of hand pose data
hideLeftboolFalseHide left hand visual but still stream data
hideRightboolFalseHide right hand visual but still stream data
disableLeftboolFalseDisable left hand tracking entirely
disableRightboolFalseDisable right hand tracking entirely

Hand Data Format

The HAND_MOVE event returns data in the following format:

typescript
type HandsData = {
  left?: Float32Array;    // 25 * 16 values (25 joints, 4x4 matrices)
  right?: Float32Array;   // 25 * 16 values
  leftState: HandState;
  rightState: HandState;
};

type HandState = {
  pinch: boolean;
  squeeze: boolean;
  tap: boolean;
  pinchValue: number;
  squeezeValue: number;
  tapValue: number;
};

Hand Landmarks

Following the XR Hand API conventions, 25 joints are tracked per hand:

JointIndexJointIndex
wrist0middle-finger-phalanx-distal13
thumb-metacarpal1middle-finger-tip14
thumb-phalanx-proximal2ring-finger-metacarpal15
thumb-phalanx-distal3ring-finger-phalanx-proximal16
thumb-tip4ring-finger-phalanx-intermediate17
index-finger-metacarpal5ring-finger-phalanx-distal18
index-finger-phalanx-proximal6ring-finger-tip19
index-finger-phalanx-intermediate7pinky-finger-metacarpal20
index-finger-phalanx-distal8pinky-finger-phalanx-proximal21
index-finger-tip9pinky-finger-phalanx-intermediate22
middle-finger-metacarpal10pinky-finger-phalanx-distal23
middle-finger-phalanx-proximal11pinky-finger-tip24
middle-finger-phalanx-intermediate12

SSL Requirement

WebXR hand tracking requires HTTPS. Use ngrok or localtunnel for development:

Learn More

For detailed examples of using Hands, see: