Look on Demand: A Cognitive Scheduling Framework for Visual Evidence Acquisition in Multimodal Reasoning

Existing multimodal reasoning approaches predominantly follow two paradigms: converting visual inputs into text prior to reasoning, or performing end-to-end reasoning within a unified vision–language representation space. Despite their empirical progress, both paradigms suffer from fundamental structural limitations. The former relies on static visual-to-text conversion, which tends to compress and lose fine-grained visual details. The latter is prone to linguistic dominance induced by joint optimization and attention mechanisms, leading to systematically weakened faithfulness to visual evidence during reasoning. In this work, we argue that a central challenge is how and when visual evidence is introduced into the reasoning process. Motivated by this insight, we propose CSMR, a multimodal reasoning framework in which a language model controls the reasoning process by deciding when to invoke an independent visual perception module to acquire task-relevant visual evidence. Experiments across multiple multimodal reasoning benchmarks show that CSMR consistently outperforms representative baseline methods in accuracy under a zero-shot setting. Further experimental analysis confirms that these advantages primarily arise from the proposed cognitive scheduling mechanism.

Answering complex questions about images requires AI systems to reason faithfully over visual evidence. Existing methods often either translate the image into text before reasoning, or directly reason over image and text inputs together. The first strategy can lose important visual details, while the second may rely too heavily on language patterns rather than the image itself. To address this, we propose CSMR, a framework that allows the model to actively look back at the image during reasoning. When the model finds that its current information is insufficient, it queries a visual module for relevant evidence and then continues reasoning based on the newly acquired information. This helps the model reduce text-based guessing and produce answers that are more faithful to the image. Experiments on multiple multimodal reasoning benchmarks show that CSMR improves performance without requiring additional training. Our analysis further shows that the improvement mainly comes from its ability to decide when visual evidence is needed.