dLLM-Cache: Accelerating Diffusion Large Language Models with Adaptive Caching

Autoregressive Models (ARMs) have long dominated the landscape of Large Language Models. Recently, a new paradigm has emerged in the form of diffusion-based Large Language Models (dLLMs), which generate text by iteratively denoising masked segments. This approach has shown significant advantages and potential. However, dLLMs suffer from high inference latency. Traditional ARM acceleration techniques, such as Key-Value caching, are incompatible with dLLMs due to their bidirectional attention mechanism. To address this specific challenge, our work begins with a key observation that dLLM inference involves a static prompt and a partially dynamic response, where most tokens remain stable across adjacent denoising steps. Based on this, we propose dLLM-Cache, a training-free adaptive caching framework that combines long-interval prompt caching with partial response updates guided by feature similarity. This design enables efficient reuse of intermediate computations without compromising model performance. Extensive experiments on representative dLLMs, including LLaDA 8B and Dream 7B, show that dLLM-Cache achieves up to *9.1*$\times$ FLOPs reduction on LongBench-HotpotQA while maintaining competitive output quality. Notably, our method brings dLLM inference latency close to that of ARMs under many settings. *The code for this work is publicly available at: https://github.com/maomaocun/dLLM-cache*.

Diffusion-based language models are a newer type of AI text model: they begin with masked words and refine the whole answer over several rounds, instead of writing one word at a time. This lets them look at both earlier and later words, but it also slows generation. At each round, the model often repeats nearly the same calculations for the prompt and for answer words that barely changed since the previous round. We built dLLM-Cache to reuse these calculations during generation, without retraining the model. The method keeps prompt calculations for many rounds because the prompt is fixed, and it refreshes answer calculations only for the words whose internal signals have changed the most. A cheap similarity check tells the system which words need fresh computation and which can safely use cached results. On LLaDA 8B and Dream 7B, dLLM-Cache reduces computation by more than nine times in the best case while keeping performance close to the original models on math, knowledge, coding, and long-context tasks. This makes diffusion language models easier to run in practice, especially when prompts are long, and brings their speed closer to standard chat models.