Plan for Speed: Dilated Scheduling for Masked Diffusion Language Models

Masked diffusion language models (MDLMs) promise fast, non-autoregressive text generation, yet existing samplers, which pick tokens to unmask based on model confidence, ignore interactions when unmasking multiple positions in parallel and effectively reduce to slow, autoregressive behavior. We propose the Dilated Unmasking Scheduler (DUS), an inference-only, planner-model-free method that partitions sequence positions into non-adjacent dilated groups and unmasks them in parallel so as to minimize an upper bound on joint entropy gain at each denoising step. By explicitly trading off the number of network calls against generation quality, DUS recovers most of the performance lost under traditional parallel unmasking strategies. Across math (GSM8K, MATH500), code (HumanEval, MBPP), general-knowledge (BBH, MMLU-Pro), and instruction following (IFEval) benchmarks, DUS outperforms confidence-based planners and turns the diffusion-specific quality-speed trade-off into a deterministic, predictable speedup set by the block size $B$, yielding up to $5.8\times$ wall-clock speedup over token-by-token MDLM decoding without modifying the underlying denoiser. Applied as a drop-in post-filter, dilated spacing also improves adaptive samplers. Code is available at https://github.com/omerlux/DUS.

Modern AI language models generate text one word at a time, which is reliable but slow. A newer family of models, called masked diffusion language models, promises much faster generation by filling in many missing words at once, like solving a crossword with many blanks simultaneously. In practice, however, existing methods pick which blanks to fill based on how confident the model is about each word, and end up filling in neighboring words together. Nearby words depend on each other heavily (the second word of a sentence is shaped by the first), so guessing them in parallel produces incoherent text and forces the model back into slow word-by-word generation. We introduce the Dilated Unmasking Scheduler (DUS), a simple rule for choosing which blanks to fill first. Instead of relying on confidence, DUS spaces its picks far apart across the sentence and then fills in the gaps in a coarse-to-fine pattern. Because the simultaneously chosen blanks are far apart, they depend on each other much less, so the model can fill them in parallel with minor loss of quality. We prove this works under standard assumptions about language and confirm it on math, coding, reasoning, and instruction-following benchmarks: DUS makes diffusion language models up to 5.8x faster than word-by-word decoding without retraining the model. The same spacing rule also improves other recent speed-up techniques when applied as a drop-in filter.