Context Forcing: Consistent Autoregressive Video Generation with Long Context

Recent approaches to real-time long video generation typically employ streaming tuning strategies, attempting to train a long-context student using a short-context (memoryless) teacher. In these frameworks, the student performs long rollouts but receives supervision from a teacher limited to short 5-second windows. This structural discrepancy creates a critical **student-teacher mismatch**: the teacher's inability to access long-term history prevents it from guiding the student on global temporal dependencies, effectively capping the student's context length. To resolve this, we propose **Context Forcing**, a novel framework that trains a long-context student via a long-context teacher. By ensuring the teacher is aware of the full generation history, we eliminate the supervision mismatch, enabling the robust training of models capable of long-term consistency. To make this computationally feasible for extreme durations (e.g., 2 minute), we introduce a context management system that transforms the linearly growing context into a **Slow-Fast Memory** architecture, significantly reducing visual redundancy. Extensive results demonstrate that our method enables effective context lengths exceeding 20 seconds—$6\text{--}10\times$ longer than state-of-the-art methods like LongLive and Infinite-RoPE. By leveraging this extended context, Context Forcing preserves superior consistency across long durations, surpassing state-of-the-art baselines on various long video evaluation metrics.

AI can now generate video in real time, but these systems tend to forget what they created moments earlier, so over a long clip, faces, objects, and backgrounds slowly drift and stop matching up. The cause lies in how the AI is trained: one model (the "student") learns to produce long videos by imitating a second model (the "teacher"), yet the teacher can only watch a few seconds at a time and remembers nothing before that. A teacher with no memory simply cannot teach the student to stay consistent over the long run. We fix this by giving the teacher a memory. Our method, Context Forcing, lets the teacher see the entire video generated so far, so it can properly guide the student to stay consistent over time. To keep this practical for videos up to two minutes long, we add a memory system that keeps recent moments in sharp detail while compressing older ones and discarding redundant information. The result is video that stays coherent over much longer stretches, roughly six to ten times longer than the best existing methods. Characters and scenes hold together from start to finish.