PVDepth: Panoramic Video Depth Estimation via Geometry-Aware Spatiotemporal Adaptation

Panoramic video depth estimation is pivotal for applications such as Virtual Reality and World Models. However, advancements in this field are impeded by two primary obstacles: the scarcity of large-scale training data and the unique spatiotemporal challenges of Equirectangular Projection (ERP), which hinder the direct transfer of perspective models. In this paper, we first present PanoCARLA, a large-scale synthetic RGB-D panoramic video dataset, featuring natural motion trajectories and drone-like roaming perspectives. Building on this foundation, we propose PVDepth, an end-to-end framework adapted from perspective video depth models. To tackle ERP-specific geometric distortions and consequent non-linear temporal dynamics, we introduce two core mechanisms: (1) A Progressive Sphere-aware Noise Initialization strategy that anneals the noise distribution from planar to spherical, guiding the model to adapt to non-uniform information density; and (2) A Cube-rectified Temporal Modeling module that incorporates an auxiliary cubemap temporal branch to rectify non-linear temporal dynamics in the ERP domain. Extensive experiments demonstrate that PVDepth achieves superior performance, generating geometrically accurate and temporally consistent depth sequences. Code and data will be released at https://github.com/ChuanxinSong/PVDepth.

Existing video depth estimation models perform poorly in the panoramic domain, where videos capture a full 360-degree view and have different geometric properties from standard perspective videos. To address this, we construct a large-scale RGB-D panoramic video dataset. Based on this dataset, we fine-tune an existing perspective video depth model, and further optimize its architecture from geometry-aware spatiotemporal perspectives according to the characteristics of panoramic videos. Our method effectively improves the accuracy of panoramic video depth estimation. This can benefit applications such as immersive 3D scene reconstruction, virtual reality, and controllable panoramic video generation.