CACR: Reinforcing Temporal Answer Grounding in Instructional Video via Candidate-Aware Causal Reasoning

The task of temporal answer grounding in instructional videos (TAGV), which aims to locate precise video segments that respond to natural language queries, is increasingly important for direct video answer retrieval. This task remains challenging due to the need to comprehend semantically complex questions and to address the significant length mismatch between untrimmed videos and short target moments. Existing methods often suffer from sensitivity to irrelevant content or insufficient visual reasoning capabilities. To tackle these limitations, we propose a Candidate-Aware Causal Reasoning (CACR) framework. Our approach first employs a Visual-Language Pre-training based Candidate Selection (VBCS) algorithm to efficiently generate K candidate segments, then applies a temporal logic reasoning module enhanced by a rejection reward mechanism and optimized via Group Relative Policy Optimization (GRPO) for robust inference. Extensive experiments on six benchmarks demonstrate that our method achieves state-of-the-art performance in terms of mean Intersection-over-Union (mIoU), providing a new perspective for reasoning-based retrieval in long videos.

Imagine watching a long instructional video — say, a 30-minute car repair tutorial — and asking, "How do I replace the brake pads?" The exact answer might be just a 20-second clip somewhere in the middle. Finding that clip automatically is surprisingly hard for AI: the question can be phrased in many ways, the relevant moment is tiny compared to the full video, and most of the footage is unrelated chatter or other steps. Existing systems often get distracted by irrelevant content or fail to understand the question well enough to point to the right place. We built a system called CACR that approaches this problem like a careful viewer. First, it quickly skims the whole video and picks out a small set of candidate segments that look like plausible answers. Then it reasons about each candidate the way a person might — checking whether it actually answers the question, and confidently rejecting it when it does not. We trained the reasoning step using a recent reinforcement-learning technique that rewards the model both for landing on the correct moment and for knowing when to say "no." Tested on six diverse benchmarks — covering tutorials, vehicle manuals, and medical procedures — CACR consistently locates the right segment more accurately than prior methods. This brings us closer to AI assistants that can give you a precise answer from a long video, rather than asking you to scrub through it yourself, which could meaningfully help in education, technical support, and medical training.