LaRI: Layered Ray Intersections for Single-view 3D Geometric Reasoning

We present Layered Ray Intersections (LaRI), a fully supervised method for occluded geometry reasoning from a single image. Unlike conventional depth estimation, which is limited to visible surfaces, LaRI predicts multiple surfaces intersected by the camera rays using layered point maps. Compared to the existing approaches that leverage neural implicit representations or iterative refinement, LaRI achieves complete scene reconstruction in one feed-forward pass, enabling efficient and view-aligned geometric reasoning to underpin both object-level and scene-level tasks. We further propose to predict the ray stopping index, which identifies valid intersecting pixels and layers from LaRI's output. To better underpin and evaluate this task, we build an annotation pipeline using rendering engines, construct annotations for five public datasets, including synthetic and real-world data covering 3D objects and scenes. As a generic method, LaRI's performance is validated in object-level and scene-level reconstruction tasks.

Understanding the full 3D structure of an object or scene from a single image is challenging because many surfaces are hidden from view. Most existing methods only estimate the visible parts of a scene, which limits their ability to reason about occluded geometry. We introduce Layered Ray Intersections (LaRI), a method that predicts multiple 3D surfaces along each camera ray instead of only the first visible surface. This allows the model to infer hidden structures from a single image in a single feed-forward pass, making reconstruction more efficient and complete. We also propose a mechanism to determine which predicted surfaces are physically valid. To support this research, we built a large-scale annotation pipeline using graphical rendering engines and created training data for five public datasets covering both objects and indoor scenes. Experiments show that LaRI improves 3D reconstruction quality for both object-level and scene-level tasks, helping machines better understand the hidden geometry of the world.