Thinking in Structures: Evaluating Spatial Intelligence in Constraint-Governed Spaces

Spatial intelligence is crucial for vision--language models (VLMs), yet many scene-centric benchmarks evaluate unconstrained environments where a single image may admit multiple plausible 3D interpretations. We introduce SSI-Bench, a VQA benchmark for Structure-Centric Spatial Reasoning (SCSR) in constraint-governed spaces. Built from complex real-world 3D structures, it uses structural constraints from geometry, topology, and physical feasibility to make component relations more determinate from visual evidence. The benchmark contains 1,000 ranking questions spanning geometric and topological reasoning, where correct ordering requires resolving all candidate-wise 3D relations, imposing stronger demands on spatial understanding. It is created through a fully human-centered pipeline with over 400 researcher-hours of image curation, component annotation, and question design. Evaluating 31 VLMs reveals a large gap to humans: the best open-source model achieves 22.2% accuracy and the strongest closed-source model reaches 33.6%, while humans score 91.6%. Further results show that chain-of-thought reasoning brings only marginal gains, and error analysis reveals fundamental limitations in current models' spatial understanding within constraint-governed spaces. Project page: https://ssi-bench.github.io.

People can often look at a photo of a bridge, roof, stairway, tower, or frame and understand how its parts are arranged in 3D, even when some parts are hidden, crowded, or seen from an unusual angle. This kind of spatial understanding is important for AI systems that need to interpret the physical world, but it is still difficult to measure reliably. In this paper, we introduce SSI-Bench, a new test designed to evaluate whether AI systems that understand both images and text can reason about real-world 3D structures. Instead of asking simple questions about objects in everyday scenes, SSI-Bench focuses on structured spaces where parts are connected and constrained by shape, geometry, and physical feasibility. The questions ask models to compare and rank highlighted parts or groups, such as which part is higher, longer, farther away, or more closely connected in the structure. We carefully built the test through human annotation and review to ensure that the answers are clear and require genuine spatial reasoning rather than simple visual shortcuts. When we tested 31 leading AI systems, even the strongest models performed far below humans. Additional “thinking” steps helped only slightly, and many errors came from misunderstanding object parts, depth, connections, or the overall 3D layout. These results show that current AI systems still lack robust spatial understanding, and SSI-Bench provides a useful tool for tracking future progress.