StructMamPose: From Sequential Perception to Structural Reasoning for 3D Human Pose Estimation
Abstract
Accurately modeling complex temporal and topological dependencies and depth information is critical for monocular 3D human pose estimation, yet existing Mamba-based approaches struggle to fulfill these demands, suffering from internal state update confusion induced by forced sequence flattening and depth modeling confusion inherent to single-view observations. To address this confusion, we propose a StructMamPose framework equipped with Identity Anchoring Mechanism (IAM) and View Transformation Hub (VTH). The IAM injects spatiotemporal identities into the parameter generation network to anchor the selectivity of state update matrices, suppressing spurious connections to enforce feature propagation along valid topological dependencies. The VTH performs internal coordinate rotation to transform implicit depth inference into observable planar features, endowing the model with explicit spatial understanding and multi-view constraints. Experimental results demonstrate that our framework achieves SOTA performance on most datasets.
Lay Summary
Estimating a person's 3D pose from a regular 2D video is extremely challenging. The main difficulties are that a single camera cannot easily perceive depth, and AI models often get confused about how human body parts are physically connected when they process video data. To overcome these issues, we developed a new AI framework called StructMamPose. Our approach introduces two key innovations. First, it explicitly teaches the AI the physical rules of the human skeleton, preventing it from making illogical connections (such as incorrectly linking a hand to a foot). Second, it equips the AI with the ability to internally imagine a "virtual side view" of the person. This allows the model to double-check and correct depth errors as if it were walking around the person, all without needing an actual second camera. By successfully giving the AI this structural awareness and virtual multi-view perspective, our method achieves highly accurate 3D human pose estimations and outperforms existing state-of-the-art models.