Quant VideoGen: Auto-Regressive Long Video Generation via 2-Bit KV-Cache Quantization

Despite rapid progress in auto-regressive video diffusion, we identify an emerging system–algorithm bottleneck that limits both deployability and generation quality: KV-cache memory. In auto-regressive video generation models, the KV-cache grows with generation history and quickly dominates GPU memory (often ≥30 GB), preventing deployment on widely available hardware. More critically, memory-bounded KV budgets constrain the effective working memory, directly degrading long-horizon consistency in identity, layout, and motion. To address this challenge, we present Quant VideoGen (QVG), a training-free KV-cache quantization framework for auto-regressive video diffusion models. QVG exploits video’s inherent spatiotemporal redundancy via Semantic-Aware Smoothing, producing low-magnitude, quantization-friendly residuals. Building on this, QVG introduces Progressive Residual Quantization, a coarse-to-fine multi-stage scheme that further reduces quantization error while enabling a smooth quality–memory trade-off. Across LongCat-Video, HY-WorldPlay, and Self-Forcing, QVG establishes a new Pareto frontier between quality and memory efficiency, reducing KV memory by up to 7.0× with less than 4% end-to-end latency overhead, while delivering significantly better generation quality than existing baselines.

Modern video generation models are becoming increasingly powerful, but generating long videos remains difficult. Many of these models create video step by step, while keeping a memory of the frames and details they have already produced. As the video gets longer, this memory becomes very large and can take up most of the GPU, making the model hard to run on common hardware. When the model is forced to use a smaller memory budget, video quality can also suffer: characters may change appearance, scene layouts may drift, and motion may become less consistent over time. We introduce Quant VideoGen, or QVG, a training-free method that makes this video memory much smaller without retraining the model. QVG takes advantage of the fact that neighboring video regions are often similar in space and time. It first reorganizes the stored information so that the remaining differences are easier to compress, then uses a multi-stage quantization method to reduce memory while preserving important visual details. Across several auto-regressive video diffusion models, QVG reduces KV-cache memory by up to 7.0× with less than 4% extra latency, while maintaining better video quality than existing memory-saving methods.