GemDepth: Geometry-Embedded Features for 3D-Consistent Video Depth

Video depth estimation extends monocular prediction into the temporal domain to ensure coherence. However, existing methods often suffer from spatial blurring in fine-detail regions and temporal inconsistencies. We argue that current approaches, which primarily rely on temporal smoothing via Transformers, struggle to maintain strict 3D geometric consistency—particularly under rotations or drastic view changes. To address this, we propose GemDepth, a framework built on the insight that an explicit awareness of camera motion and global 3D structure is a prerequisite for 3D consistency. Distinctively, GemDepth introduces a Geometry-Embedding Module (GEM) that predicts inter-frame camera poses to generate implicit geometric embeddings. This injection of motion priors equips the network with intrinsic 3D perception and alignment capabilities. Guided by these geometric cues, our Alternating Spatio-Temporal Transformer (ASTT) captures latent point-level correspondences to simultaneously enhance spatial precision for sharp details and enforce rigorous temporal consistency. Furthermore, GemDepth employs a data-efficient training strategy, effectively bridging the gap between high efficiency and robust geometric consistency. As shown in Fig.2, comprehensive evaluations demonstrate that GemDepth achieves state-of-the-art performance across multiple datasets, particularly in complex dynamic scenarios.

When artificial intelligence tries to turn regular 2D videos into 3D environments, it often struggles. If the camera moves too fast or turns sharply, the AI gets confused, resulting in blurry details and flickering video frames. This happens because the AI lacks a true, built-in understanding of real-world 3D space. To solve this, we created GemDepth, a new AI system that acts like it has a built-in 3D compass. Instead of just guessing and smoothing out pixels, our program explicitly tracks how the camera is moving from one frame to the next. By feeding these camera movement clues directly into the network, it learns to accurately match specific physical points over time. As a result, GemDepth can generate sharp, stable, and flicker-free 3D videos, even in complex, fast-moving scenes. Furthermore, our highly efficient training strategy sets a new standard for accurate 3D video understanding.