Learn to Think: Improving Multimodal Reasoning through Vision-Aware Self-Improvement Training

Post-training with explicit reasoning traces is common to improve the reasoning capabilities of Multimodal Large Language Models (MLLMs). However, acquiring high-quality reasoning traces is often costly and time-consuming. Hence, the self-improvement paradigm has emerged, enabling MLLMs to self-generate reasoning traces for training without external supervision. Despite its effectiveness, we reveal two shortcomings in the self-improvement training of MLLMs: 1) data imbalance, where simple samples are over-trained, but the challenging yet crucial samples are under-trained; 2) language prior bias, where MLLMs overly rely on linguistic priors while neglecting the visual cues. To this end, we propose VISTA, a VIsion-aware Self-improvement Training framework for enhancing the multimodal Reasoning of MLLMs. Specifically, VISTA first introduces a prefix resampling strategy to reuse the partial correct reasoning traces for efficient data collection, and then designs a vision-aware attention score to quantify the model’s focus on visual information. Extensive experiments show that VISTA can be applied to various post-training scenarios, i.e., supervised fine-tuning and preference learning, and effectively enhances the multimodal reasoning performance across various MLLMs and tasks, e.g., bringing up to +13.66% average performance gains for Qwen2.5-VL-3B-Instruct.

Multimodal AI models can reason about both images and text — for example, interpreting a medical scan or solving a geometry problem from a diagram. A cost-effective way to improve these models is self-improvement: letting them practice generating step-by-step reasoning and learning from their own correct answers. However, we found two problems with this approach: the model mostly practices easy questions while struggling on harder ones, and it often ignores what it actually sees in the image, instead relying on language patterns to guess the answer — a form of "visual hallucination." We propose VISTA, a framework that addresses both issues. To tackle the imbalance, we recycle partially correct reasoning from failed attempts as a starting point for generating new solutions to difficult questions. To combat hallucinations, we measure how much attention the model pays to the image during reasoning, filtering out solutions where the model barely looks at the visual content. Across five models and five tasks, VISTA improves reasoning accuracy by up to 13.66%, while also reducing hallucinations and improving the model's ability to generalize to new, unseen tasks.