Accelerating Video Inverse Problem Solvers with Autoregressive Diffusion Models

Autoregressive video diffusion models enable low-latency generation by producing videos chunk by chunk, yet their potential for measurement-conditioned long-horizon video restoration remains underexplored. Existing diffusion-based video inverse solvers typically restore all frames holistically, delaying the first output frame until the entire video is reconstructed, and require repeated VAE passes for measurement consistency. We propose AVIS, a measurement-conditioned autoregressive video diffusion framework for streaming restoration. AVIS starts reverse diffusion from a measurement-consistent initialization and restores each chunk autoregressively with KV-cache conditioning. We further introduce AVIS Flash, which applies measurement guidance only to the first chunk and restores later chunks through guidance-free autoregressive propagation from the corrected prefix. On five video restoration tasks, AVIS reduces initial latency from 114s to 4s and improves throughput from 0.71 to 1.18 FPS compared to a leading non-autoregressive solver, while achieving stronger restoration quality. AVIS Flash further increases throughput to 5.91 FPS on a single RTX 4090 GPU, offering a practical efficiency--quality trade-off for real-time long-horizon video restoration.