VideoDiffCR: Content-Aware Adaptive Token Pruning for Autoregressive Video Diffusion Transformers

Causal autoregressive video diffusion models enable streaming generation by producing video chunks sequentially and reusing a rolling KV cache. However, even in this efficient setting, every denoising step still processes all space–time tokens in the current chunk through every transformer block, resulting in substantial redundant computation. We present VideoDiffCR , a differentiable token-pruning framework for causal video diffusion transformers. Building on DiffCR, our method learns layer- and timestep-adaptive token retention ratios, but introduces video-specific changes required for autoregressive generation: a temporal mixing router that scores tokens using cross-frame context, an alternating initialization of layerwise retention ratios for stable training, and a denoising-step schedule that prunes more aggressively at noisier steps while preserving tokens at later refinement steps. We apply VideoDiffCR to a Self Forcing-distilled Wan2.1 1.3B video diffusion model and train it using only the DMD distillation signal on VidProM text prompts, without video supervision. On VBench, VideoDiffCR maintains performance close to the dense Self Forcing baseline while achieving up to 1.91× end-to-end speedup, showing that learned token pruning can substantially improve the efficiency of causal video diffusion models.