AgentVidBench: A Multi-Hop Video Question Answering Benchmark for Evaluating MLLM Agents

Autonomous MLLM agents are emerging to solve challenging real-world video understanding tasks that require multi-hop multimodal reasoning, yet there is a critical absence of video benchmarks equipped to rigorously evaluate their agentic capabilities. To bridge this gap, we introduce AgentVidBench, a multi-hop video question answering benchmark focused on evaluating the spatial, temporal, and causal reasoning capabilities of MLLM agents. Beyond standard question-answer pairs, AgentVidBench provides structured evidence milestones for trajectory evaluation, exposing trace-level failures hidden behind terminal success and providing actionable signals for improving agent behavior. Our trace-level diagnostics categorize these failures into five axes: Task Understanding, Evidence Coverage, Evidence Grounding, Evidence Completeness, and Reasoning Faithfulness. Experiments with 12 MLLMs and four agentic frameworks on AgentVidBench show that while agentic workflows improve standard accuracy, they still frequently succumb to premature fixation and incomplete evidence verification. We further present a simple yet effective agentic strategy that mitigates these failures through flexible, task-adaptive evidence-seeking in a single reasoning loop. Code and datasets are available at https://anonymous.4open.science/r/avb-icmlw.