Blind Frames, Broken Stories: the Verification Bottleneck in Scientific Video Generation
Abstract
Generative AI can automate the conversion of academic papers into short educational videos by combining LLM orchestration, text-to-image generation, image-to-video synthesis, and speech generation. This workflow, however, requires reliable verification of scientific figures before they are animated. We propose the Agentic Director Pipeline, a multi-stage system for producing 45-75 second scientific videos with explicit factual and visual verification gates. Using a diagnostic corpus of 70 scientific figures with controlled error injections, we evaluate the cross-modal verification stage with five contemporary Vision-Language Models: GPT-5.1, Claude Sonnet 4, Gemini 2.5 Pro, Qwen3.6-VL, and DeepSeek-VL2. Zero-shot and Chain-of-Thought prompting are insufficient: models over-reject correct figures at rates of up to 0.86 and miss up to 0.86 of injected errors. We identify three recurring failure patterns — Knowledge Contamination, Spatial Insensitivity, and the Chain-of-Thought Paradox — and use them to motivate a verifier architecture based on grounded structural schemes and adversarial Advocate-Critic checking.