Norm-Controlled Likelihood Guidance for Diffusion-based Inverse Solver

Diffusion-based inverse solvers approximate the posterior by combining a pretrained diffusion prior with an approximate likelihood guidance term. Across diverse host solvers, we identify a consistent diagnostic signal: for the same image and inverse task, different noise realizations lead to different reconstructions, and larger likelihood-guidance norms reliably predict worse perceptual quality. This observation motivates us to propose the likelihood-score norm as a general, quantitative error proxy. To leverage this proxy, we propose Norm-Controlled Likelihood Guidance, a set of three lightweight, host-agnostic modules that steer the sampling trajectory toward smaller likelihood norms while remaining on the diffusion manifold. Theoretically, we justify our motivation by proving that under a tractable multi-modal model, smaller likelihood norms can reliably imply smaller score approximation errors. Experimentally, we demonstrate that the proposed method can acheive consistent improvements across multiple diffusion-based solvers on FFHQ and ImageNet-256 datasets. Ablation studies and cost analyses further validate the design and show favorable quality–compute trade-offs.