Perceptual Flow Network for Visually Grounded Reasoning

Despite the success of LVLMs, general optimization objectives (e.g., standard MLE) fail to constrain visual trajectories, leading to language bias and hallucination. To mitigate this, current methods introduce geometric priors from visual experts as additional supervision. However, we observe that such supervision is typically suboptimal: it is biased toward geometric precision and offers limited reasoning utility. To bridge this gap, we propose Perceptual Flow Network (PFlowNet), which eschews rigid alignment with the expert priors and achieves interpretable yet more effective visual reasoning. Specifically, PFlowNet decouples perception from reasoning to establish a self-conditioned generation process. Based on this, it integrates multi-dimensional rewards with vicinal geometric shaping via variational reinforcement learning, thereby facilitating reasoning-oriented perceptual behaviors while preserving visual reliability. PFlowNet delivers a provable performance guarantee and competitive empirical results, particularly setting new SOTA records on V* Bench (90.6%) and MME-RealWorld-lite (67.0%).

Large Vision-Language Models (LVLMs) can answer visual questions, but they may guess when the needed evidence is small, subtle, or surrounded by clutter. A typical solution is to make LVLMs look at boxes proposed by object detectors, with the assumption that the tightest box around an object provides the best evidence. We show that this assumption can fail: a very tight crop may remove useful context, causing LVLMs to see the object clearly but misunderstand the scene. We propose Perceptual Flow Network that teaches an LVLM to first plan what it needs to inspect, choose one or more helpful image views, describe them, and then answer based on those views. Rather than simply imitating the detector prior, PFlowNet learns to prefer views that are both visually reliable and useful for solving the question. This also makes the reasoning process more transparent, because the selected regions and their descriptions can be inspected before the final answer. In tests on general visual question answering and fine-grained tasks involving small objects or spatial relations, PFlowNet improves accuracy and sets new best results on several challenging benchmarks. Our results suggest that stronger visual reasoning is not only about looking at the correct object, but also about choosing the right view of the evidence.