CoD-Lite: Real-Time Diffusion-Based Generative Image Compression

Recent advanced diffusion methods typically derive strong generative priors by scaling diffusion transformers. However, scaling fails to generalize when adapted for real-time compression scenarios that demand lightweight models. In this paper, we explore the design of real-time and lightweight diffusion codecs by addressing two pivotal questions. First, does diffusion pre-training benefit lightweight diffusion codecs? Through systematic analysis, we find that generation-oriented pre-training is less effective at small model scales whereas compression-oriented pre-training yields consistently better performance. Second, are transformers essential? We find that while global attention is crucial for standard generation, lightweight convolutions suffice for compression-oriented diffusion when paired with distillation. Guided by these findings, we establish a one-step lightweight convolution diffusion codec that achieves real-time 60 FPS encoding and 42 FPS decoding at 1080p. Further enhanced by distillation and adversarial learning, the proposed codec reduces bitrate by 85% at a comparable FID to MS-ILLM, bridging the gap between generative compression and practical real-time deployment. Codes are released at https://github.com/microsoft/GenCodec/tree/main/CoD_Lite.

Modern generative diffusion models have shown great promise in compressing images by using generative technology to recover compressed contents. However, these diffusion codecs are usually too large and slow to be used in real-time, everyday applications. Our research tackles this bottleneck by answering two key questions: how should we train smaller diffusion models for compression, and do they really need to be complex? We discovered that instead of training the model on general image generation, training it specifically for compression yields much better results at a smaller scale. Furthermore, we proved that simpler, traditional computing blocks can replace bulky attention structures without losing efficiency. Based on these insights, we developed a lightweight diffusion codec that runs at lightning speed, processing high-definition (1080p) content smoothly in real time. It reduces data transmission needs by 85% compared to leading methods while keeping the images looking sharp and realistic, paving the way for practical, real-time generative compression.