Pair2Scene: Learning Local Object Relations for Procedural Scene Generation

Generating high-fidelity 3D indoor scenes remains a significant challenge due to data scarcity and the complexity of modeling intricate spatial relations. Current methods often struggle to scale beyond training distribution to dense scenes or rely on LLMs/VLMs that lack the ability for precise spatial reasoning. Building on top of the observation that object placement relies mainly on local dependencies instead of information-redundant global distributions, in this paper, we propose Pair2Scene, a novel procedural generation framework that integrates learned local rules with scene hierarchies and physics-based algorithms. These rules mainly capture two types of inter-object relations, namely support relations that follow physical hierarchies, and functional relations that reflect semantic links. We model these rules through a network, which estimates spatial position distributions of dependent objects conditioned on position and geometry of the anchor ones. Accordingly, we curate a dataset 3D-Pairs from existing scene data to train the model. During inference, our framework can generate scenes by recursively applying our model within a hierarchical structure, leveraging collision-aware rejection sampling to align local rules into coherent global layouts. Extensive experiments demonstrate that our framework outperforms existing methods in generating complex environments that go beyond training data while maintaining physical and semantic plausibility.

Creating realistic 3D indoor scenes, like bedrooms or offices, is a difficult task for AI because it requires a deep understanding of how objects relate to one another in space. Most current methods try to learn the layout of an entire room at once, but they often run out of data or fail when trying to generate crowded, complex scenes with many small items. In this paper, we introduce "Pair2Scene," a new approach that changes how AI thinks about room design. Instead of trying to manage the whole room at once, our system focuses on "local pairs." It learns simple rules based on how two objects interact: for example, a laptop usually sits on top of a desk (a support relation), and a keyboard is typically placed right in front of a laptop (a functional relation). By breaking a complex room down into these small, understandable pairs, our AI can "build" a scene piece-by-piece. We also developed a large new dataset called "3D-Pairs" to teach the AI these specific relationships. Our results show that Pair2Scene can create much richer and more detailed environments than previous methods. It doesn't just copy what it saw in training; it can combine its learned rules to generate rooms that are even more complex and realistic than the ones it was originally shown. This technology could help game developers, filmmakers, and robot designers automatically create high-quality virtual worlds that look and feel like real human spaces.