Absorbing Quantization Error by Deformable Noise Scheduler for Diffusion Models

Diffusion models deliver state-of-the-art image quality but are expensive to deploy. Post-training quantization (PTQ) can shrink models and speed up inference, yet residual quantization errors distort the diffusion distribution (the timestep-wise marginal over $\boldsymbol{x}_t$), degrading sample quality. We propose a distribution-preserving framework that absorbs quantization error into the generative process without changing architecture or adding steps. Deformable Noise Scheduler (DNS) reinterprets quantization as a principled timestep shift, mapping the quantized prediction distribution $\boldsymbol{x}_t$ back onto the original diffusion distribution so that the target marginal is preserved. Unlike trajectory-preserving or noise-injection methods limited to stochastic samplers, our approach preserves the distribution under both stochastic and deterministic samplers and extends to flow-matching with Gaussian conditional paths. It is plug-and-play and complements existing PTQ schemes. Empirically, our method consistently enhances generation quality across diverse backbones and existing PTQ baselines. Notably, when further quantizing the FP16 LoRA branch of SVDQuant to enable fully integer inference, our approach effectively mitigates the performance drop, reducing FID from 27.16 to 26.22. Code is available at https://github.com/ZephyrYoung-eYuan/DNS_AQE

Today's best AI image generators—known as diffusion models—produce stunning pictures, but running them requires expensive, power-hungry hardware. A common way to make them faster and cheaper is quantization, which compresses the model (like shrinking a high-resolution photo to a smaller file). Unfortunately, this compression introduces small errors at each step of the image generation process, and these errors accumulate, noticeably degrading the final image quality. In this paper, we introduce a simple yet effective fix. Rather than trying to prevent the compression errors, we adjust the generation schedule so the errors are naturally absorbed along the way—like a driver who, instead of fighting potholes on the road, slightly shifts their route so the bumps don't matter. Our method, called Deformable Noise Scheduler (DNS), requires no changes to the model itself and adds almost no extra computation. It is a plug-and-play solution: you can apply it on top of any existing quantized diffusion model and immediately see better images.