Lottery Prior: Randomized Neural Compression for Zero-Shot Inverse Problems

We study zero-shot inverse problems, where a clean signal is recovered from a single degraded observation without external training data. Contrary to the common belief that such problems require highly complex models, we show that a lightweight neural network, when combined with entropy and complexity regularization in a compression-based formulation, is sufficient for high-quality restoration. We propose Lottery Prior, a compression-based inverse solver that leverages architectural priors from random networks and induces a family of implicit priors through randomness, enabling ensemble-based refinement. We further derive non-asymptotic error bounds for compression-based maximum-likelihood inverse solvers, revealing how rate–distortion constraints act as implicit regularizers. Experiments on denoising, noisy super-resolution, and inpainting demonstrate that our method achieves state-of-the-art with significantly fewer effective parameters. Project page: https://eedavidwu.github.io/LotteryPrior/

Many inverse problems aim to recover a clean signal from a corrupted one, such as removing noise, increasing resolution, or filling missing regions for images. Existing methods often rely on large trained models, which may be unavailable in scientific, medical, or specialized imaging with limited training data. This paper shows that a surprisingly small random neural network, guided by a compression principle, can act as an effective prior using only the corrupted observation itself. Our method, Lottery Prior, searches for simple explanations of the observation and combines multiple random guesses when helpful. Experiments show that it can restore images accurately while using far fewer effective parameters than existing zero-shot methods.