SpatioLM: Towards General Physical Spatial Intelligence in Vision-Language Models

Vision-Language Models (VLMs) perform well on commonsense reasoning tasks but struggle with visual spatial reasoning. Most existing solutions introduce extra 3D prior inputs or external spatial encoders, which increase complexity and degrade the underlying VLMs' general-purpose capabilities after spatial fine-tuning. To this end, we propose a parameter-efficient \textit{\textbf{Spatio}-vision \textbf{L}anguage \textbf{M}odels (SpatioLM)}, that enhances spatial intelligence without extra 3D prior inputs or third-party spatial encoders. Concretely, we design a plug-and-play and non-invasive spatio-vision module that elicits the spatial knowledge inherent in VLMs. Furthermore, we innovatively leverage pseudo depth and camera information as supervision to guide the model in learning physically coherent representations. Extensive experiments show that SpatioLM achieves significant improvements in diverse tasks, including spatial perception and understanding while effectively limiting the degradation of general capabilities. Notably, the model achieves an impressive score of 71.6 on the VSI-Bench (the first model to surpass 70). In addition, it attains competitive performance when transferred to embodied manipulation tasks.

AI systems that understand both images and text are good at recognizing objects and answering everyday questions, but they often struggle to understand physical space: where things are, how far away they are, and how objects relate to one another in a scene. This matters for tasks such as navigation, scene understanding, and robot control. We introduce SpatioLM, a lightweight add-on that helps existing image-and-language AI systems reason better about space without needing special 3D sensors, extra 3D inputs, or separate spatial expert models. During training, SpatioLM learns from simple cues about depth and camera direction, helping the model build a more physically grounded understanding of images. Across a wide range of tests, SpatioLM improves spatial reasoning while preserving the model’s general abilities. It is also the first model to score above 70 on VSI-Bench, a challenging spatial-reasoning benchmark, and transfers well to robot manipulation tasks.