Compressed Image Generation with Denoising Diffusion Codebook Models

We present a novel generative approach based on Denoising Diffusion Models (DDMs), which produces high-quality image samples along with their losslessly compressed bit-stream representations. This is obtained by replacing the standard Gaussian noise sampling in the reverse diffusion with a selection of noise samples from pre-defined codebooks of fixed iid Gaussian vectors. Surprisingly, we find that our method, termed Denoising Diffusion Codebook Model (DDCM), retains sample quality and diversity of standard DDMs, even for extremely small codebooks. We leverage DDCM and pick the noises from the codebooks that best match a given image, converting our generative model into a highly effective lossy image codec achieving state-of-the-art perceptual image compression results.More generally, by setting other noise selections rules, we extend our compression method to any conditional image generation task (e.g., image restoration), where the generated images are produced jointly with their condensed bit-stream representations.Our work is accompanied by a mathematical interpretation of the proposed compressed conditional generation schemes, establishing a connection with score-based approximations of posterior samplers for the tasks considered.Code and demo are available on our project's website.

We developed a new method for generating high-quality images using AI, while also creating a compact, compressed version of each generated image at the same time. This is useful because we can store or send these images more efficiently without losing quality. Our approach builds on diffusion models, a popular type of GenAI models, which usually rely on random noise to create images.Instead of using random noise, we select the noise from a fixed set of options — like choosing from a finite menu of available noises. Then, to re-create the exact same generated image later on, we only need to tell which items from the menu were selected, and re-select them.We leverage this property and propose a powerful method for compressing images, beating previous compression methods especially when aiming for small file sizes. We also show how this technique can be used for other tasks like restoring damaged images, all while keeping the stored data compact.