MotionMAR: Multi-scale Auto-Regressive Human Motion Reconstruction from Sparse Observations

Human motion follows a temporal hierarchical structure, transitioning from low-frequency global trajectories to high-frequency details. Inspired by the success of multi-level autoregressive models in computer vision, we propose MotionMAR, a coarse-to-fine framework for motion reconstruction from sparse observations. It first estimates the global trajectory of human motion and then gradually refines the temporal details. This architecture consists of four integrated components. The Temporal Multi-scale Tokenization (TMT) VQ-VAE encodes the data at multiple temporal resolutions, separating semantic motion from minor jitters. The Motion Autoregressive Network (MAN) operates in this latent space, predicting motion across scales. It first establishes the global structure through coarse indices and then generates finer indices to recover specific details. Meanwhile, the Scale-Aware Control (SAC) module integrates sparse tracking data to ensure the generated output aligns with actual observations. The Motion Refinement Network (MRN) subsequently smooths consecutive poses and eliminates quantization artifacts. Experiments show that MotionMAR achieves state-of-the-art accuracy on the AMASS dataset, providing a reliable and structure-aware approach for motion reconstruction. The source code is publicly available at \url{http://www.lidarhumanmotion.net/motionmar/}.

From the human perception perspective, we first focus on the broader human motion trajectory and then on the high-frequency motion details. Previous human motion reconstruction methods typically represent human motion as a single motion sequence and directly restore the motion based on this sequence. Why not decompose human motion into multiple levels, including the overall motion trajectory and local motion details, and then restore the human motion from coarse to fine? Therefore, we introduce MotionMAR, a coarse-to-fine framework for motion restoration from sparse observations (a few sensors). We design the Temporal Multi-scale Tokenization method to split human motion into multiple levels of action segments over time. The motion autoregressive network reconstructs action segments at different temporal scales from coarse to fine, ensuring the consistency of the overall trajectory and local details. The MotionMAR proposed provides a theoretically grounded and biologically inspired method for motion modeling in virtual reality and augmented reality applications.