Are VLMs Seeing or Just Saying? Uncovering the Illusion of Visual Re-examination

Vision-Language Models (VLMs) often produce self-reflective statements like “let me check the figure again” during reasoning. Do such state- ments trigger genuine visual re-examination, or are they merely learned textual patterns? We in- vestigate this via VISUALSWAP, an image-swap probing framework: after a model reasons over an image, we replace it with a visually similar but semantically different one and test whether the model notices. We introduce VS-BENCH, 800 image pairs curated from MathVista, Math- Verse, MathVision, and MMMU-Pro. Exper- iments on Qwen3-VL, Kimi-VL, and ERNIE- VL reveal a striking failure: models overwhelm- ingly miss the swap, with accuracy dropping by up to 60%. Counterintuitively, thinking mod- els are nearly 3x more vulnerable than their in- structed counterparts, and scaling offers no mit- igation. Multi-turn user instructions restore vi- sual grounding, but self-generated reflective state- ments during continuous generation do not. At- tention analysis explains why: user instructions substantially elevate attention to visual tokens, whereas self-reflection does not. Current VLMs tend to say rather than actually see when claiming to perform visual re-examination. Our code and dataset are available at the project page: https://visualswap.github.io/

Modern AI systems that can look at images and answer questions about them often "narrate" their thinking out loud. In the middle of solving a problem, they might say things like "Wait, let me look at the picture again to make sure." This sounds reassuring—as if the AI is double-checking its work by actually re-examining what it sees. But is it really looking again, or just saying the words? We designed a simple test to find out. After an AI looks at a picture and starts reasoning about it, we quietly swap the picture for a very similar one that has a small but important difference—one that changes the correct answer. Then we let the AI continue, including the moment where it claims to "check the image again." If it were truly looking, it would notice the change and update its answer. We built a collection of 800 such picture pairs (covering math diagrams, charts, and other visual puzzles) to run this test fairly across many AI systems. The result was striking: the AIs almost always failed to notice the swap. They kept reasoning as if the original picture were still there, even right after announcing they would re-examine it—with accuracy dropping by as much as 60%. Surprisingly, the newer "thinking" models that reason at greater length were nearly three times worse than simpler ones, and making models bigger didn't help. We traced the cause: when an AI tells itself to look again, it barely shifts its focus back to the image. But when a human user explicitly tells it to look again, its focus on the image jumps and its accuracy largely recovers—showing the ability to see was there all along; the model just won't trigger it on its own. This matters because people may trust an AI's stated "I checked carefully" in important settings like reading medical scans or guiding self-driving systems. Our work shows that such reassurances can be hollow, and points toward building AI that genuinely looks again rather than merely claiming to.