Shape of Thought: Progressive Object Assembly via Visual Chain-of-Thought

Multimodal models for text-to-image generation have achieved strong visual fidelity, yet they remain brittle under compositional structural constraints—notably generative numeracy, attribute binding, and part-level relations. To address these challenges, we propose Shape-of-Thought (SoT), a visual CoT framework for process-supervised progressive shape assembly in the rendered 2D domain, without external engines at inference time. SoT trains a unified multimodal autoregressive model to generate interleaved textual plans and rendered intermediate states, helping the model capture shape-assembly logic without producing explicit geometric representations. Unlike text-only CoT, each decision is grounded in a rendered state, making counts, attachments, topology, and intermediate part-addition errors inspectable across the trajectory. To support this paradigm, we introduce SoT-26K, a large-scale dataset of grounded assembly traces derived from part-based CAD hierarchies, and T2S-CompBench, a benchmark for evaluating structural integrity and trace faithfulness. Fine-tuning on SoT-26K achieves 88.4\% on component numeracy and 84.8\% on structural topology, outperforming direct generation by +24.2 points on component numeracy and +19.3 points on structural topology. SoT establishes a transparent testbed for rendered-domain structure-aware generation. The code is available at https://github.com/yuhuo03/Shape-of-Thought.

AI systems can now create impressive pictures from text, but they still often struggle when the request describes a precise object structure: for example, the exact number of parts, where handles or legs should attach, or how small components should fit together. This paper studies how to make such generation more reliable and easier to inspect. We introduce Shape-of-Thought, a method that asks the model to build an object step by step instead of drawing the final result all at once. At each step, the model writes a short description of what it is adding and then produces an updated picture of the object. This is similar to watching an object being assembled piece by piece. Because the intermediate pictures are visible, users can see where the model is following the plan and where it may make a mistake. To train and test this idea, we create SoT-26K, a collection of about 26,000 object assembly examples built from objects with known parts and converted into step-by-step picture sequences. We also design tests that check whether generated objects have the right number of parts, correct connections, and plausible overall structure. Our results show that step-by-step visual assembly improves structural accuracy compared with generating the final image directly. The work provides a more transparent way to study controllable object generation in rendered images, rather than claiming to fully solve editable 3D model creation.