GeoSense: Internalizing Geometric Necessity Perception for Multimodal Reasoning

Advancing towards artificial superintelligence requires rich and intelligent perceptual capabilities. A critical frontier in this pursuit is overcoming the limited spatial understanding of Multimodal Large Language Models (MLLMs), where geometry information is essential. Existing methods often address this by rigidly injecting geometric signals into every input, while ignoring their necessity and adding computation overhead. Contrary to this paradigm, our framework endows the model with an awareness of perceptual insufficiency, empowering it to autonomously engage geometric features in reasoning when 2D cues are deemed insufficient. To achieve this, we first introduce an independent geometry input channel to the model architecture and conduct alignment training, enabling the effective utilization of geometric features. Subsequently, to endow the model with perceptual awareness, we curate a dedicated spatial-aware supervised fine-tuning dataset. This serves to activate the model’s latent internal cues, empowering it to autonomously determine the necessity of geometric information. Experiments across multiple spatial reasoning benchmarks validate this approach, demonstrating significant spatial gains without compromising 2D visual reasoning capabilities, offering a path toward more robust, efficient and self-aware multi-modal intelligence.

We aim to help multimodal AI systems develop a better understanding of three-dimensional space. Although current vision and language models can answer many questions about images, they still often struggle to judge distance, direction, occlusion, object size, and spatial relationships. These abilities are especially important for robotics, navigation, and assistive perception. This paper introduces GeoSense, a method that teaches the model to decide when three-dimensional geometric information is truly needed, instead of forcing it to use such information for every question. For ordinary image understanding tasks, the model can rely on the original visual content. For questions that require spatial reasoning, it can use additional geometric information. We construct training samples from public datasets so that the model can learn which questions benefit from geometric information and which questions should avoid it. Experiments show that GeoSense performs better on multiple spatial reasoning tests while largely preserving general visual understanding. Because extra computation is triggered only when it is likely to affect the answer, GeoSense is also more efficient than methods that always use three-dimensional geometric information. This work may contribute to more reliable embodied AI systems, robots, and assistive perception systems.