ACTIVE-o3 : Empowering MLLMs with Active Perception via Pure Reinforcement Learning

Active vision, also known as active perception, refers to actively selecting where and how to look in order to gather task-relevant information. It is a critical component of efficient perception and decision-making in humans and advanced embodied agents. With the rise of Multimodal Large Language Models (MLLMs) as central planners in robotic systems, the lack of methods for equipping MLLMs with active perception has become a key gap. We first provide a systematic definition of MLLM-based active perception tasks and show that GPT-o3's zoom-in strategy can be viewed as a special case, though it suffers from low efficiency and inaccurate region selection. To address these issues, we propose Active-o3, a reinforcement learning framework built on GRPO that equips MLLMs with active perception capabilities. Leveraging a modular sensing-action design and a dual-form reward, Active-o3 autonomously learns efficient and stable region selection strategies without explicit supervision. We further establish a comprehensive benchmark covering both open-world tasks (small/dense-object grounding) and domain-specific scenarios (remote sensing, autonomous driving, interactive segmentation). Experimental results demonstrate that Active-o3 significantly enhances active perception capabilities compared to Qwen2.5-VL-CoT. Moreover, we show that our RL framework not only preserves the model’s general understanding ability but can also serve as a proxy task for leveraging perception data, further improving performance on benchmarks such as RealWorldQA. We hope that our work can provide a simple codebase and unified evaluation protocol to facilitate future research on active perception with MLLMs.

This paper studies how vision-language AI systems can learn to look more actively, rather than relying only on a single full image. In many real-world images, important objects can be tiny, crowded, or easy to miss. Humans often solve this by first scanning the scene and then looking more closely at promising regions. We aim to give multimodal AI models a similar ability. We propose Active-O3, a training framework that teaches a model to choose useful image regions to zoom in on before making its final prediction. The model learns this behavior through trial and feedback, without requiring humans to manually label which regions it should inspect. We also build a benchmark covering difficult cases such as small objects, dense scenes, aerial images, driving scenes, and fine-grained segmentation. Our experiments show that Active-O3 helps models find and understand small or hard-to-see objects more accurately than passive baselines. The results suggest that teaching AI systems where to look can make visual perception more efficient, reliable, and useful for future applications such as robotics, autonomous systems, and visual assistants.