CountsDiff: A diffusion model on the natural numbers for generation and imputation of count-based data

Diffusion models have excelled at generative tasks for both continuous and token-based domains, but their application to discrete ordinal data remains underdeveloped. We present CountsDiff, a diffusion framework designed to model distributions on the natural numbers. CountsDiff extends the Blackout diffusion framework by simplifying its formulation through a direct parameterization in terms of a survival probability schedule and an explicit loss weighting. This introduces flexibility through design parameters with direct analogues in existing diffusion modeling frameworks. Beyond this reparameterization, CountsDiff introduces features from modern diffusion models, previously absent in counts-based domains, including continuous-time training, classifier-free guidance, and churn/remasking reverse dynamics that allow non-monotone reverse trajectories. We propose an initial instantiation of CountsDiff and validate it on natural image datasets (CIFAR-10, CelebA), exploring the effects of the introduced design parameters in a complex, well-studied, and interpretable data domain. We then highlight biological count assays as a natural use case, evaluating CountsDiff on single-cell RNA-seq imputation in fetal and heart cell atlases. Remarkably, we find that even this simple instantiation matches or surpasses the performance of a state-of-the-art discrete generative model and leading scRNA-seq imputation methods, while leaving substantial headroom for further gains through optimized design choices in future work.

Modern AI has transformed how we create, manipulate, and understand complex data. However, modern AI methods are typically designed for images or text, and aren’t well suited for data represented by counts. Counts are prevalent across scientific measurements, from counting how many times each certain gene is expressed in a cell to counting photons in sensors. Existing methods either treat counts as decimals or treat each possible value as a distinct category; both adaptations result in unexpected and undesirable results. CountsDiff is a diffusion model, the technology behind most modern image and video generation AI, built from the ground up to model whole numbers. Instead of blurring data with white noise, like standard diffusion models, CountsDiff progressively removes individual units one by one. It then learns to rebuild the original count, starting from zero. We validate CountsDiff on images, then show a potential application to single-cell RNA sequencing, a method that counts how many times genes are expressed in cells. These data are full of “dropouts,” where genes that are expressed are missed and falsely recorded as zero. CountsDiff can fill in these missing values while preserving the natural variability of gene expression, which can provide biologists with better data for analysis. Beyond this, CountsDiff provides a framework to use AI to model any data resulting from counting as opposed to measuring.