4RC: 4D Reconstruction via Conditional Querying Anytime and Anywhere

We present 4RC, a unified feed-forward framework for 4D reconstruction from monocular videos. Unlike existing methods that typically decouple motion from geometry or produce limited 4D attributes, such as sparse trajectories or two-view scene flow, 4RC learns a holistic 4D representation that jointly captures dense scene geometry and motion dynamics. At its core, 4RC introduces a novel encode-once, query-anywhere and anytime paradigm: a transformer backbone encodes the entire video into a compact spatio-temporal latent space, from which a conditional decoder can efficiently query 3D geometry and motion for any query frame at any target timestamp. To facilitate learning, we represent per-view 4D attributes in a minimally factorized form, decomposing them into base geometry and time-dependent relative motion. Extensive experiments demonstrate that 4RC outperforms prior methods across a wide range of 4D reconstruction tasks.

Real-world scenes are not static: people move, and objects interact. However, most AI-enabled computer vision systems either reconstruct the 3D shape of a scene without fully understanding how it moves, or track motion without building a complete 3D representation. We introduce 4RC, a model that reconstructs both the 3D structure of a scene and how it changes over time from an ordinary video. The key idea is to let the model watch the whole video once, build a compact understanding of the scene, and then answer flexible questions, such as where a point, object, or surface is at any moment in time. This makes it possible to recover not only scene geometry but also dense 3D motion within one unified system. This research can help machines better understand dynamic real-world environments. This ability is important for applications such as robotics, augmented and virtual reality, video editing, and future AI systems that need to reason about how the physical world moves and changes.