Find, Fix, Reason: Context Repair for Video Reasoning

Reinforcement learning has advanced video reasoning in large multi-modal models, yet dominant pipelines either rely on on-policy self-exploration, which plateaus at the model’s knowledge boundary, or hybrid replay that mixes policies and demands careful regularization. Dynamic context methods zoom into focused evidence but often require curated pretraining and two-stage tuning, and their context remains bounded by a small model’s capability. In contrast, larger models excel at instruction following and multi-modal understanding, can supply richer context to smaller models, and rapidly zoom in on target regions via simple tools. Building on this capability, we introduce an observation-level intervention: a frozen, tool-integrated teacher identifies the missing spatiotemporal dependency and provides a minimal evidence patch (e.g., timestamps, regions etc.) from the original video while the question remains unchanged. The student answers again with the added context, and training updates with a chosen-rollout scheme integrated into Group Relative Policy Optimization (GRPO). We further propose a Robust Improvement Reward (RIR) that aligns optimization with two goals: outcome validity through correct answers and dependency alignment through rationales that reflect the cited evidence. Advantages are group-normalized across the batch, preserving on-policy exploration while directing it along causally meaningful directions with minimal changes to the training stack. Experiments on various related benchmarks show consistent accuracy gains and strong generalization. Project page is available at https://jethrojames.github.io/FFR/.

Videos often require understanding when events happen, where objects are, and how actions unfold over time. Find, Fix, Reason (FFR) improves video question answering by using a stronger teacher model to help only when a student model makes a mistake. The teacher does not reveal the answer, but points to useful evidence in the original video, such as relevant frames or regions. The student then tries again with this context, and training rewards evidence-based corrections. This helps models perform better on video tasks that require temporal, spatial, and causal reasoning.