Skipping the Zeros in Diffusion Models for Sparse Data Generation

Diffusion models (DMs) excel on dense continuous data, but are not designed for sparse continuous data. They do not model exact zeros that represent the deliberate absence of a signal. As a result, they erase sparsity patterns and perform unnecessary computation on mostly zero entries. With Sparsity-Exploiting Diffusion (SED), we model only non-zero values, preserving sparsity. SED delivers computational savings while maintaining or improving generation quality by skipping zeros during training and inference. Across physics and biology benchmarks, SED matches or surpasses conventional DMs and domain-specific baselines, while vision experiments provide intuitive insights into the limitations of dense DMs and the benefits of SED.

Many scientific datasets are sparse, meaning that most entries are exactly zero and only a small fraction contain non-zero values. For example, in particle detectors, only a few regions record energy, and in biology, only some genes are active in a given cell. These empty entries are not mistakes—they often mean that nothing happened there, and this absence can be scientifically important. However, many modern AI models for generating new data treat every entry as if it might contain a signal, which wastes computation and can turn meaningful zeros into small artificial values. We introduce Sparsity-Exploiting Diffusion (SED), a generative AI method that skips over empty parts of the data and focuses only on the values that are actually present. This lets the model preserve the pattern of what is absent and what is present, while using less computation. Across experiments in particle physics, biology, and image data, SED produces realistic sparse samples and is especially efficient when the data has many zeros. The method is less advantageous when the data is only mildly sparse. Still, it shows that generative AI can better match the structure of scientific data by treating the absence of data as meaningful.