Image Restoration via Diffusion Models with Dynamic Resolution

Diffusion models (DMs) have exhibited remarkable efficacy in various image restoration tasks. However, existing approaches typically operate within the high-dimensional pixel space, resulting in high computational overhead. While methods based on latent DMs seek to alleviate this issue by utilizing the compressed latent space of a variational autoencoder, they require repeated encoder-decoder inference. This introduces significant additional computational burdens, often resulting in runtime performance that is even inferior to that of their pixel-space counterparts. To mitigate the computational inefficiency, this work proposes projecting data into lower-dimensional subspaces using dynamic resolution DMs to accelerate the inference process. We first fine-tune pre-trained DMs for dynamic resolution priors and adapt DPS and DAPS, which are two widely used pixel-space methods for general image restoration tasks, into the proposed framework, yielding methods we refer to as SubDPS and SubDAPS, respectively. Given the favorable inference speed and reconstruction fidelity of SubDAPS, we introduce an enhanced variant termed SubDAPS++ to further boost both reconstruction efficiency and quality. Empirical evaluations across diverse image datasets and various restoration tasks demonstrate that the proposed methods outperform recent DM-based approaches in the majority of experimental scenarios. The code is available at https://github.com/StarNextDay/SubDAPS.git.

Damaged, blurry, incomplete, or low-quality images often need to be restored before they can be useful. Recent restoration methods can produce impressive restorations, but many of these restoration methods are slow and may require a large amount of computer memory because these methods process the image at full detail throughout the entire restoration process. The entire restoration process is often unnecessary. In the early stages, the model mainly needs to recover the overall structure of the image, while fine details become important later. In ouf work, we develop a faster image restoration approach that first works with smaller, lower-resolution restoration process of the image and then gradually increases the resolution as the image becomes clearer. The dynamic resolution sampling process allows the model to focus on broad structure first and refine details later, reducing wasted computation. We apply the idea of dynamic resolution sampling to several restoration tasks. Across multiple datasets and tasks, our method restores images with quality comparable to or better than recent approaches while using less time and memory.