DyLLM: Efficient Diffusion LLM Inference via Saliency-based Token Selection and Partial Attention

Masked diffusion language models enable parallel token decoding, providing a promising alternative to the sequential nature of autoregressive generation. However, their iterative denoising process remains computationally expensive because it repeatedly processes the entire sequence at every step. We observe that across these diffusion steps, most token representations remain stable; only a small subset, which we term *salient tokens*, contributes meaningfully to the next update. Leveraging this temporal sparsity, we present **DyLLM**, a training-free inference framework that accelerates decoding by selectively computing only these salient tokens. DyLLM identifies saliency by measuring the cosine similarity of attention contexts between adjacent denoising steps. It recomputes feed-forward and attention operations only for salient tokens while reusing cached activations for the remainder. Across diverse reasoning and code-generation benchmarks, DyLLM achieves up to 9.6 $\times$ higher throughput while largely preserving the baseline accuracy of representative open-source diffusion LLMs, LLaDA and Dream.

Some newer language models generate text by gradually filling in missing words, rather than writing one word at a time from left to right. This approach can generate multiple words in parallel, but it is computationally expensive because the model repeatedly revisits the entire sentence at every step. We found that most parts of the sentence change very little between these repeated steps, while only a small number of important tokens need careful updating. DyLLM uses this observation to speed up generation: it focuses computation on the tokens that changes a lot, and reuses previously computed results for the rest. This allows the model to avoid much of the repeated work while still preserving the quality of the generated text. Across reasoning and code-generation benchmarks, DyLLM achieves up to 9.6 $\times$ higher throughput on representative open-source diffusion language models while largely preserving accuracy. By making diffusion-based language models faster and more practical, this work helps reduce the computational cost of using large language models.