PhyScene3D: Physically Consistent 3D Interactive Tabletop Scene Generation

Generating physically consistent 3D tabletop scenes is a fundamental yet underexplored problem for interactive and generalist robotic learning. The challenge stems from dense object hierarchies and irregular affordances. Here, an interactive scene denotes a physically valid, collision-free environment directly loadable into physics simulators. Existing methods, ranging from decoupled symbolic solvers to end-to-end regression models, often suffer from error propagation or overfitting to noisy supervision containing widespread physical violations. To address these limitations, we introduce PhyScene3D, a framework that reformulates generation as a Human-Mimetic Constructive Process. The proposed Cognitive Topological Reasoning Chain (CTRC) factorizes scene synthesis into a sequential, anchor-conditioned process. It employs a 3D AABB-based placement scheme that imposes a strong structural inductive bias. To address imperfect supervision and physical infeasibility, we introduce Physics-Aware Denoising Alignment (PADA). It integrates a differentiable Signed Distance Field (SDF) with Test-Time Optimization (TTO) to project generated scenes onto a physics-feasible manifold while preserving semantic intent. Experiments demonstrate that PhyScene3D outperforms state-of-the-art approaches in both semantic accuracy and physical validity, achieving a 40\% reduction in scene-wise collision rate relative to the human-annotated training data.

Training robots to handle cluttered desks requires vast amounts of realistic 3D environments. Yet generating these scenes automatically is hard because everyday tabletop setups involve complex physical relationships. A pen must sit inside a holder, books must stack without falling, and a keyboard must rest beside a monitor. Existing methods fall short in two ways. Symbolic planners often lack physical common sense and produce impossible layouts. Data-driven models simply copy errors from human annotations, reproducing floating or intersecting objects. We introduce PhyScene3D, which generates scenes the way a person would assemble a real desk. It plans hierarchically, placing supports before resting items and containers before their contents. It then runs an internal physics check to detect and fix collisions. Instead of blindly replicating imperfect training data, the system learns to correct it. Our results show that PhyScene3D outperforms prior approaches and even exceeds its human training data, cutting collision rates by 40%. The generated scenes are immediately ready for physics simulators used in robotic learning.