ScDiVa: Masked Discrete Diffusion for Joint Modeling of Single-Cell Identity and Expression

Single-cell RNA-seq profiles are high-dimensional, sparse, and unordered, causing autoregressive generation to impose an artificial ordering bias and suffer from error accumulation. To address this, we propose scDiVa, a masked discrete diffusion foundation model that aligns generation with the dropout-like corruption process by defining a continuous-time forward masking mechanism in token space. ScDiVa features a bidirectional denoiser that jointly models discrete gene identities and continuous values, utilizing entropy-normalized serialization and a latent anchor token to maximize information efficiency and preserve global cell identity. The model is trained via depth-invariant time sampling and a dual denoising objective to simulate varying sparsity levels while ensuring precise recovery of both identity and magnitude. Pre-trained on 59 million cells, scDiVa achieves strong transfer performance across major benchmarks, including batch integration, cell type annotation, and perturbation response prediction. These results suggest that masked discrete diffusion serves as a biologically coherent and effective alternative to autoregression.

Single-cell sequencing allows scientists to see how individual cells behave, but the data are messy and unordered, making it hard for standard AI models to learn from them.We developed a new approach called scDiVa, which treats the measurement of each gene like a token that can be randomly masked and then recovered. This mimics how real biological data contain missing or noisy measurements. Our model learns to simultaneously predict which genes are turned on and at what level, without forcing an artificial order onto the cells.We trained scDiVa on 59 million cells, and it outperformed existing methods on key tasks such as merging datasets from different experiments, identifying cell types, and predicting how cells respond to perturbations.This work provides a more biologically natural way for AI to understand single-cell data, which could accelerate research in drug discovery, disease mechanisms, and personalized medicine.