d3LLM: Ultra-Fast Diffusion LLM using Pseudo-Trajectory Distillation

Diffusion large language models (dLLMs) offer capabilities beyond those of autoregressive (AR) LLMs, such as parallel decoding and random-order generation. However, realizing these benefits in practice is non-trivial, as dLLMs inherently face an *accuracy-parallelism trade-off*. Despite increasing interest, existing methods typically focus on only one-side of the coin, targeting either efficiency or accuracy. To address this limitation, we propose d3LLM (*Pseudo-Distilled Diffusion Large Language Model*), striking a balance between accuracy and parallelism: (i) during training, we introduce *pseudo-trajectory distillation* to teach the model which tokens can be decoded confidently at early steps, thereby improving parallelism; (ii) during inference, we employ *entropy-based multi-block decoding* with a KV-cache refresh mechanism to achieve high parallelism while maintaining accuracy. To better evaluate dLLMs, we also introduce AUP (*Accuracy Under Parallelism*), a new metric that jointly measures accuracy and parallelism. Experiments demonstrate that our d3LLM achieves up to $10\times$ speedup over vanilla LLaDA/Dream, and $5\times$ speedup over AR models without much accuracy drop. Our code is available at https://github.com/hao-ai-lab/d3LLM.

Diffusion language models serve as a promising alternative to conventional AI models, as they can generate multiple words simultaneously rather than one at a time, but going faster typically hurts accuracy. In this work, we propose d3LLM, which tackles this trade-off from both ends: during training, we teach the model to mimic the order in which a teacher model generates words, so it learns which words it can confidently predict early; during inference, we decode multiple text segments in parallel while periodically correcting errors. The result is up to 10× speedup over existing diffusion models and 5× over conventional models, with negligible accuracy loss.