CFPO: Counterfactual Policy Optimization for Multimodal Reasoning

Large Vision-Language Models (LVLMs) have demonstrated remarkable capabilities in multimodal reasoning. However, prevailing reinforcement learning (RL) paradigms lack explicit counterfactual enhancement and causal learning mechanisms. This fundamental deficiency results in severe grounding failures, manifesting as a tendency to ignore visual evidence in favor of language priors or exhibiting hallucination drift during long chain-of-thought reasoning. To address this root cause, we propose CounterFactual Policy Optimization (CFPO), a novel framework that enforces causal consistency between visual perception and textual reasoning. CFPO introduces a cross-modal counterfactual enhancement mechanism, which regularizes the policy by maximizing the discrepancy between the model’s predictions and those from a counterfactual state where critical visual cues are suppressed. This approach seamlessly integrates with standard algorithms like GRPO and DAPO without requiring external reward models or additional supervision. Extensive experiments demonstrate that CFPO significantly improves reasoning fidelity, achieving consistent gains of 3.17%-6.25% over standard RL baselines and 1.32%-2.13% over the state-of-the-art perception-aware method (PAPO). Code is available at https://github.com/Raven-July/CFPO.

Today's AI models can process both text and images, but they often ignore visual evidence and guess based on familiar text patterns, leading to incorrect, hallucinated answers. To fix this, we introduce CounterFactual Policy Optimization (CFPO), a new training method that ensures the AI actually relies on the image. During training, we test the AI by hiding the most important visual clues — not by crudely blurring the input image, but by precisely suppressing those clues deep inside the model's internal attention mechanism. If the model gives the same answer without these crucial details, the system recognizes it is just guessing and penalizes it. This forces the AI to ground its reasoning in real visual facts rather than language shortcuts. Experiments across ten diverse benchmarks show that CFPO significantly reduces hallucinations and improves reasoning accuracy by 3。17%–6.25% over standard baselines and 1.32%–2.13% over previous state-of-the-art methods, with particularly strong gains on mathematical and real-world visual tasks.