ERGeoBench: A Comprehensive Benchmark for Embodied Reasoning and Geo-localization in Multimodal Large Language Models

Multimodal large language models (MLLMs) have shown strong potential as embodied agents, yet embodied geo-localization remains underexplored due to the lack of fine-grained evaluation. We introduce ERGeoBench, a diagnostic benchmark for vision-driven embodied geo-localization. ERGeoBench evaluates models under three progressive settings---single-view, panorama-view, and embodied-view---where agents may actively acquire observations through sequential changes in yaw, pitch, and zoom. The benchmark contains 2,207 globally distributed street-view panoramas and measures four complementary capabilities: foundational perception, spatial awareness, common sense reasoning, and geo-localization reasoning. Evaluations of leading proprietary and open-source MLLMs show that current models can infer high-level geographic semantics, but still struggle with fine-grained perceptual operations, metric localization, and spatial consistency across views. We further observe that geo-localization is strongly correlated with the other capability dimensions, suggesting that accurate localization depends on integrated perception, spatial reasoning, and commonsense inference rather than isolated visual recognition. Overall, ERGeoBench provides a unified framework for diagnosing and advancing human-like embodied geo-localization. Project Page: \url{https://kaixuewen.github.io/ERGeoBench/}

How can multimodal large language models figure out where they are in the world from what they see? Existing geo-localization benchmarks usually ask models to make a prediction from a single image or a full panorama, but this does not reflect how humans often solve the task: we look around, focus on useful clues, and update our guess step by step. In this paper, we introduce ERGeoBench, a benchmark for evaluating whether multimodal models can perform this kind of active, human-like geo-localization. The benchmark contains globally distributed street-view panoramas and tests models under three settings: a single fixed view, a panoramic view, and an embodied view where the model can rotate, look up or down, and zoom in to gather more evidence. ERGeoBench also evaluates several abilities that are important for localization, including visual perception, spatial awareness, common-sense reasoning, and final geographic prediction. Our experiments show that today’s models can often recognize high-level geographic cues, but still struggle with fine-grained perception, spatial consistency, and precise location estimation. ERGeoBench provides a diagnostic tool for building more reliable location-aware multimodal agents.