STABLE: Simulation-Ready Tabletop Layout Generation via a Semantics–Physics Dual System

Generating simulation-ready tabletop scenes from task instructions is an intriguing and promising research direction in the field of Embodied AI. However, existing task-to-scene generation methods rely exclusively on large language models (LLMs) to predict scene layouts, inevitably yielding object collisions or floating due to LLMs’ inherent limitations in 3D spatial reasoning. In this paper, we present \textbf{STABLE}, a semantics–physics dual-system tailored for simulation-ready tabletop scene generation. STABLE consists of two complementary modules: (i) a \textbf{Semantic Reasoner}, a fine-tuned LLM trained on a structured tabletop scene dataset to generate coarse layouts from input task instructions, and (ii) a \textbf{Physics Corrector}, a physics-aware flow-based denoising model that outputs pose updates to refine layouts, which ensures the physical plausibility of scenes while preserves semantic alignment with task instructions. STABLE adopts a progressive generation paradigm: by alternating between the Semantic Reasoner and Physics Corrector, it incrementally expands the scene from task-critical objects to background objects. Experiments demonstrate that STABLE successfully generates simulation-ready tabletop scenes that strictly conform to task instructions and significantly enhances the physical validity of scenes over prior art.

This paper introduces STABLE, a system that automatically creates realistic tabletop scenes for robot simulation from everyday task instructions. For example, given an instruction such as preparing a table for making tea or placing fruit on a plate, the system decides which objects should appear and where they should be placed. It then adjusts the scene so that objects do not unrealistically overlap, float in the air, or break simple physical rules. This makes the generated scenes more useful for training and testing robots in simulation. Experiments show that STABLE creates tabletop scenes that better follow task instructions and are more physically realistic than previous methods.