Continuous Viewpoint Adaptation for Single View 3D Object Reconstruction

Single-view 3D object reconstruction presents a formidable challenge in computer vision due to the inherent limitations of information obtainable from a solitary viewpoint. Recent 3D Gaussian Splatting (3DGS) inspired approaches perform a feed-forward way of learning a neural network that predicts 3D Gaussians which compose the 3D object, given a single image. However, they often struggle with occlusions and exhibit high sensitivity to small changes in input viewpoint, leading to inconsistencies and blurry artifacts in novel view renderings. Our method leverages 3DGS and introduces a new learning scheme that continuously adapts to input viewpoints. To address inherent continuity of camera viewpoints that are represented by polar and azimuthal angles, we use Neural Ordinary Differential Equations to continuously model filter subspace of neural network, thus seamlessly embedding inductive bias of perspective distortions into its structure. By continuously adapting to view-specific features, our approach fosters view consistency in 3D reconstruction, allowing better coherency and accuracy across different angles. Experiments demonstrate that our model outperforms previous methods on multiple single-view 3D reconstruction benchmark datasets and excels in extrapolating to unseen camera angles and categories.

Single-view 3D reconstruction aims to recover the full 3D shape of an object from just one image, which is difficult because many parts of the object may be hidden from view. Recent fast AI systems can generate 3D objects directly from a single image, but they often produce blurry or inconsistent results when the viewing angle changes slightly. Our work addresses this problem by designing a model that continuously adapts to the camera viewpoint. Instead of using fixed network parameters, our method smoothly changes parts of the network depending on the viewing direction, helping the model better understand how objects appear from different angles. This allows the system to produce more coherent and stable 3D reconstructions, especially in regions that are partially occluded. We evaluate our approach on several standard 3D reconstruction benchmarks and show that it improves reconstruction quality, viewpoint consistency, and generalization to unseen object categories and camera angles.