Reliable Thinking with Images

As a multimodal extension of Chain-of-Thought (CoT), Thinking with Images (TWI) has emerged as a promising avenue to enhance the reasoning capability of Multi-modal Large Language Models (MLLMs), which generates interleaved CoT by incorporating visual cues into the textual reasoning process. However, the success of existing TWI methods heavily relies on the assumption that interleaved CoTs are faultless, which is easily violated due to the complexity of multimodal understanding. In this paper, we reveal and study a highly-practical yet under-explored problem in TWI, termed Noisy Thinking (NT). Specifically, NT refers to the imperfect visual cues mining and answer reasoning process. As the saying goes, ``One mistake leads to another'', erroneous interleaved CoT would cause error accumulation, thus significantly degrading the performance. To solve NT, we propose a novel method dubbed Reliable Thinking with Images (RTWI). In brief, RTWI estimates the reliability of visual cues and textual CoT in a unified text-centric manner and then employs robust filtering and voting modules to prevent NT from contaminating final answer. Extensive experiments on seven benchmarks verify the effectiveness of RTWI against NT.

With the rapid development of reasoning-oriented AI models such as OpenAI o3 and o4-mini, Thinking with Images (TWI) has emerged as a promising avenue for improving the reasoning ability of Multimodal Large Language Models (MLLMs). Instead of relying only on text-based reasoning, TWI enables MLLMs to actively manipulate images through operations such as zooming and cropping and then facilitates step-by-step reasoning to derive answers. However, existing TWI methods often overlook the problem of Noisy Thinking (NT), where undesirable visual cues or reasoning steps would mislead the subsequent reasoning process. As a result, MLLMs may fall into the dilemma of ``One mistake leads to another''. To address this issue, we propose a plug-and-play test-time scaling method that could identify NT and then prevent NT from contaminating the final answer. Experiments on high-resolution image understanding, multimodal mathematical reasoning, and open-ended visual question answering benchmarks demonstrate that our method consistently improves reasoning accuracy and efficiency.