FOCUS: DLLMs Know How to Tame Their Compute Bound

Diffusion Large Language Models (DLLMs) offer a compelling alternative to Auto-Regressive models, but their deployment is constrained by high decoding cost. In this work, we identify a key inefficiency in DLLM decoding: while computation is parallelized over token blocks, only a small subset of tokens is decodable at each diffusion step, causing most compute to be wasted on non-decodable tokens. We further observe a strong correlation between attention-derived token importance and token-wise decoding probability. Based on this insight, we propose FOCUS, an inference system designed for DLLMs. By dynamically focusing computation on decodable tokens and evicting non-decodable ones on-the-fly, FOCUS increases the effective batch size, alleviating compute limitations and enabling scalable throughput. Empirical evaluations demonstrate that FOCUS achieves up to 3.52× throughput improvement over the production-grade engine LMDeploy in large-batch settings, while preserving or improving generation quality across multiple benchmarks.

Diffusion language models are a promising alternative to today's common word-by-word language models because they can work on many parts of an answer at the same time. In practice, however, they still spend most of their computing effort on word positions that are not ready to be finalized, so their speed stops improving when many user requests are served together. We found that a model's early attention patterns provide a useful clue about which positions are likely to be ready soon. FOCUS uses this clue during generation to keep computing on the most promising positions and temporarily skip the rest, without retraining the model. This reduces wasted work while preserving the information the model needs to produce good text. Across chat and reasoning benchmarks, FOCUS reduces the number of processed token positions by about 65% to 80% and generates text up to 3.52× faster than the LMDeploy baseline in high-concurrency settings, while maintaining or sometimes improving output quality. This matters because it makes diffusion-based language models more practical to serve in real applications, where speed, cost, and many simultaneous users are central constraints.