Seeing is Solving: Unlocking Efficient Multimodal RL via View Alignment
Abstract
Although Reinforcement Learning Fine-Tuning (RLFT) applied to Vision-Language Models (VLMs) substantially enhances multimodal reasoning capabilities, their prohibitive training cost limits broad adoption. Surprisingly, most existing methods simply port Large Language Model (LLM) RLFT techniques to VLMs, while ignoring a intrinsic property of multimodal models: their dynamic text–vision alignment. We ask a new question: Can this intrinsic alignment be turned into a training signal that makes VLM RLFT more efficient? We analyze how a VLM plans to attend, actually attends, and ideally should attend during reasoning, and derive two lightweight metrics from these patterns. Predictive View Accuracy (PVA) estimates sample difficulty, and Reasoning View Accuracy (RVA) reflects the quality of chain-of-thought (CoT) reasoning. These alignment signals enable automated data curriculum and dense reasoning supervision. We introduce FOCUS-RL, a plug-and-play framework that can be seamlessly integrated into any VLM and dramatically boosts RLFT training efficiency. FOCUS-RL achieves 2.5 x – 4 x faster convergence over vanilla GRPO and consistent accuracy gains (+4.4 on average) across six different benchmarks and multiple VLM families.
Lay Summary
Vision-language models can answer questions that involve both images and text, but improving their reasoning ability often requires costly training. This paper presents FOCUS-RL, a more efficient training framework that uses a model’s own image-text alignment as guidance. Instead of treating all training examples equally, FOCUS-RL checks whether the model is looking at the right visual information while reasoning and uses this signal to identify useful training samples and provide better feedback during learning. As a result, the model learns faster and reasons more accurately across different tasks and model families, achieving 2.5×–4× faster training convergence and consistent accuracy improvements.