The Perceptual Bandwidth Bottleneck in Vision-Language Models: Active Visual Reasoning via Sequential Experimental Design

Visual perception in modern Vision-Language Models (VLMs) is constrained by a perceptual bandwidth bottleneck: a broad field of view preserves global context but sacrifices the fine-grained details required for complex reasoning. We argue that high-resolution visual reasoning is therefore not only semantic reasoning but also task-relevant evidence acquisition under limited perceptual bandwidth. Inspired by active vision and information foraging, we formalise this process as sequential Bayesian optimal experimental design (S-BOED), where an agent decides which visual evidence to acquire before answering. Since exact Bayesian inference is intractable in continuous gigapixel spaces, we derive a tractable coverage-resolution objective as a proxy for task-relevant information gain. We instantiate this framework with FOVEA, a training-free procedure that refines VLM crop proposals through evidence-oriented probing. Experiments on high-resolution benchmarks show consistent gains over direct and ReAct-style baselines, with particularly strong improvements in search-dominated remote-sensing settings.

Modern vision-language AI systems can answer questions about images, but they often struggle when the answer depends on tiny details in very large or high-resolution images. If the system looks at the whole image at once, it keeps the global context but may lose small objects, text, or fine spatial details. If it zooms in too much, it may see details clearly but miss the relevant region entirely. This paper studies this trade-off and proposes a way for AI systems to actively choose which parts of an image to inspect before answering. Instead of treating image cropping or zooming as a simple preprocessing step, we view it as an evidence-gathering process: the model should decide where to allocate its limited visual attention in order to obtain the most useful information for the task. We introduce FOVEA, a training-free method that refines a model’s proposed image crops by testing whether different candidate regions contain enough visual evidence to answer the question. The method can be used with existing vision-language models and external visual tools. Across several high-resolution image benchmarks, FOVEA improves over direct answering and standard tool-use baselines, with especially strong gains in remote-sensing tasks where important objects are small and sparse. Overall, our results suggest that making vision-language models better at high-resolution reasoning requires not only stronger reasoning ability, but also better control over what visual evidence they choose to inspect.