SPEED: Sharpened-Teacher Distillation for Parallel Decoding of Diffusion Language Models

Diffusion-based large language models generate text by gradually filling in masked tokens, yet they remain slow because they usually decode only a few tokens per step. Parallel decoding, which unmasks multiple tokens simultaneously, promises acceleration but often degrades quality when too many tokens are predicted at once. We identify the root cause: when decoding is viewed as iterative token grouping, overly permissive grouping places interdependent tokens in the same step, violates the conditional independence assumption, and amplifies reliance on noisy context even when the top prediction is already correct. We introduce SPEED, a framework that enlarges safe parallel groups through complementary training and inference designs. At training time, a sharpened teacher distillation objective selectively aligns the student to teacher-correct positions using a temperature-scaled KL term together with a masked language modeling loss, producing a student that assigns more probability mass to correct token identities and elevates more positions above the decoding threshold. At inference time, Slow-Fast Decoding partitions tokens by sensitivity to revealed context using token-wise Jensen-Shannon Divergence computed with and without access to the preceding block, decoding low-sensitivity tokens jointly in parallel while deferring high-sensitivity tokens until sufficient context resolves them. Through extensive experiments, our framework attains up to 12.2$\times$ speedup on LLaDA-8B-Instruct and 6.7$\times$ on Dream-7B-Instruct with accuracy close to greedy decoding across standard reasoning and code benchmarks.

Diffusion-based language models write text by gradually filling in missing words, a bit like solving a sentence with many blanks. This can produce strong results, but it is often slow because the model usually fills in only a small number of blanks at each step. A natural way to make it faster is to fill in many blanks at once, but this can hurt quality when the blanks depend on each other. We propose SPEED, a method that helps these models decide which words can be safely predicted together and which should wait for more context. During training, SPEED teaches the model to become more confident on predictions that a stronger teacher model gets right. During generation, it separates easy, less context-dependent words from harder, more context-sensitive words, allowing the easy words to be filled in quickly while delaying the harder ones until the surrounding text becomes clearer. This makes diffusion language models much faster while keeping their answers close in quality to slower decoding methods. In our experiments, SPEED achieves up to 12.2× faster generation on LLaDA-8B-Instruct and 6.7× faster generation on Dream-7B-Instruct across reasoning and code tasks.