STORM: Segment, Track, and Object Re-Localization from a Single Image

Accurate 6D pose estimation and tracking are core capabilities for physical AI systems, yet real-world deployment remains brittle and labor-intensive. Many pipelines rely on CAD models, manual masking, or per-object adaptation, and still fail under occlusion or fast motion without a principled way to recognize failure. We propose STORM, a unified framework for reference-conditioned 6D tracking that can operate from a single reference image, with minimal manual input and improved robustness. STORM combines: (i) Hierarchical Spatial Fusion Attention (HSFA), a task-driven reference-query fusion architecture that supports both single-reference and multi-reference conditioning and can optionally use vision-language semantic conditioning to resolve instance ambiguities; and (ii) a BCE-trained tracking verifier whose continuous compatibility logit is used as an energy-like score to detect drift and trigger automatic re-initialization. Experiments on LM-O and YCB-Video show that STORM improves annotation-free pose tracking accuracy over strong baselines and recovers reliably from severe occlusions and rapid viewpoint changes with minimal overhead.

Robots and other physical AI systems need to know where objects are and how they move in order to interact with the world safely. Today, many object-tracking systems still depend on carefully prepared 3D models, manual object masks, or object-specific tuning, and they can fail silently when the object is briefly hidden or moves quickly. We developed STORM, a system that can start from a single reference image of an object and track that object in new scenes. STORM first compares the reference image with the current camera view to find the object, then estimates its 6D pose for tracking. It also learns to check whether tracking is still reliable, rather than blindly trusting every new frame. When STORM detects that tracking has likely drifted, it automatically re-localizes the object and resumes tracking. This makes object pose tracking more practical for dynamic settings such as robotics, logistics, and manufacturing, where objects may be occluded, viewed from new angles, or moving quickly.