Fast and Scalable Analytical Diffusion

Analytical diffusion models offer a mathematically transparent path to generative modeling by formulating the denoising score as an empirical-Bayes posterior mean. However, this interpretability comes at a prohibitive cost: the standard formulation necessitates a full-dataset scan at every timestep, scaling linearly with dataset size. In this work, we present the first systematic study addressing this scalability bottleneck. We challenge the prevailing assumption that the entire training data is necessary, uncovering the phenomenon of Posterior Progressive Concentration: the effective golden support of the denoising score is not static but shrinks asymptotically from the global manifold to a local neighborhood as the signal-to-noise ratio increases. Capitalizing on this, we propose Dynamic Time-Aware Golden Subset Diffusion (GoldDiff), a training-free framework that decouples inference complexity from dataset size. Instead of static retrieval, GoldDiff uses a coarse-to-fine mechanism to dynamically pinpoint the "Golden Subset" for inference. Theoretically, we derive rigorous bounds guaranteeing that our sparse approximation converges to the exact score. Empirically, GoldDiff achieves a 71× speedup on AFHQ while matching or achieving even better performance than full-scan baselines. Most notably, we demonstrate the first successful scaling of analytical diffusion to ImageNet-1K.

Image generators work like sculptors who start with a block of random static and gradually carve it into a clear image. Most rely on a neural network, a "black box" that works without revealing why it makes each choice. A transparent alternative exists, one that requires no training at all: it builds each step directly from the training images using a clear mathematical rule, so every decision can be traced. The bottleneck is speed: at every step it must scan all the training data, which becomes hopelessly slow when there are millions of images. We found that this exhaustive search is unnecessary and even harmful. As the image sharpens, the training images that actually shape the result shrink from all the training data down to a small neighborhood of look-alikes. Our method, GoldDiff, exploits this: at each step it intelligently homes in on just this small "golden subset" of relevant images, with mathematical guarantees that quality is preserved. GoldDiff runs 71× faster while matching or beating the exhaustive approach, and is the first to scale this transparent technique to a large -scale dataset. This puts image generators we can actually understand and trust within practical reach.