Why hand tracking dropped to ~0% in a small set of episodes

Frames from episode 0027_ET_20260520_153157_7c05f0 — one of the ~2–3% of episodes flagged with both hands near-untracked.

2.5%

left-hand coverage

0.0%

right-hand coverage

camera streams, all intact

Root cause: black nitrile gloves

The operator wore black nitrile gloves for this food-prep task. RGB hand detection keys on skin tone and finger texture; matte black gloves against a dark apron leave the tracker nothing to lock onto, from any camera angle. The video itself is fully intact — this is a tracking-input problem, not a capture defect.

Ego camera: both hands in black gloves cutting on a board — **Ego (head) camera · t=8:00** — both gloved hands mid-task; the exact frames a tracker should cover

Ego camera: gloved hands with cleaver over cutting board — **Ego (head) camera · t=15:00** — matte black gloves against a black apron

Ego camera: gloved hands early in episode — **Ego (head) camera · t=2:00** — gloves on from the start; coverage is near-zero for the full episode

Wrist camera view of gloved hand holding knife — **Left wrist camera · t=8:00** — wrist views see the hands too, but gloves defeat RGB detection from every angle

Wrist camera view of gloved hands — **Left wrist camera · t=15:00**

Environment camera showing the capture station — **Environment camera · t=8:00** — capture station context: four synced streams, all recording normally

What we're doing about it. Testing glove colors/patterns the tracker handles well and updating capture guidelines accordingly; evaluating whether the existing glove episodes can be recovered with a glove-tolerant tracking pass; and shipping per-episode left/right/either-hand coverage in delivery metadata so any low-coverage episode is visible before it ships.