GePBench: Evaluating Fundamental Geometric Perception for Multimodal Large Language Models

Geometric shapes play important roles in both physical world and human cognition. While multimodal large language models (MLLMs) have made significant advancements in visual understanding, their abilities to recognize geometric shapes and their spatial relationships, which we term geometric perception, are not explicitly and systematically explored. To address this gap, we introduce GePBench, a novel benchmark specifically designed to assess the geometric perception capabilities of MLLMs. Our extensive evaluations reveal that even the current state-of-the-art MLLMs exhibit significant deficiencies in geometric perception tasks. Furthermore, we show that models trained with GePBench data demonstrate considerable improvements on a wide range of downstream tasks, highlighting the critical role of geometric perception in enabling advanced multimodal applications. Our code and datasets are available at https://github.com/Changhao-Xiang/GePBench.

Geometric shapes and their relationships are fundamental to how humans understand the visual world. However, there has been no systematic benchmark for testing whether multimodal AI models can reliably perform this kind of geometric perception. To study this problem, we developed an automated data synthesis engine that generates controlled geometric examples and built GePBench, a benchmark for evaluating geometric perception in multimodal large language models. Our evaluations show that even state-of-the-art open-source and closed-source models often struggle with basic geometric recognition and spatial-relation tasks. We also find that mixing geometric data into training can improve model performance on a range of downstream tasks. These results highlight an important limitation of today’s multimodal AI systems: they may appear strong at visual understanding while still lacking reliable perception of basic geometry. Improving geometric perception may therefore be an important step toward building multimodal models that reason more accurately and robustly about the visual world. To support future research, we have released our code and dataset to the community.