Locally Coherent Parallel Decoding in Diffusion Language Models

Diffusion language models (DLMs) have emerged as a promising alternative to autoregressive (AR) models, offering sub-linear generation latency and bidirectional capabilities that are particularly appealing for code generation and editing. Achieving sub-linear latency in discrete DLMs requires predicting multiple tokens in parallel. However, standard DLMs sample tokens independently from conditional marginal distributions, failing to capture the joint dependencies among concurrently generated tokens. As a result, they often lead to syntactic inconsistencies and break multi-token structures. In this work, we introduce CoDiLA (Coherent Diffusion with Local Autoregression), a method that reconciles parallel sampling with local dependency modeling. Rather than forcing the DLM to resolve fine-grained syntax, CoDiLA delegates local decoding to a small, auxiliary AR model operating on the diffusion latents. This design allows for parallel generation while ensuring sequential validity within a block and maintaining core DLM capabilities, including bidirectional modeling across blocks. We demonstrate that using a highly compact auxiliary AR model (e.g., 0.6B parameters) effectively eliminates coherence artifacts, establishing a new Pareto frontier for accuracy and speed in code generation benchmarks.

Current AI assistants generate text one word at a time, making them slow when writing long passages. A newer technology called "diffusion" speeds this up by generating entire sections of text in just a few steps. However, because it generates all the words simultaneously without checking how they connect, it often produces flawed, unusable text. To fix this, we created CoDiLA. Our system uses a fast diffusion model to instantly sketch a high-level global plan for the text. Then, a small, highly specialized AI acts as a local "executor." It takes this rough sketch and carefully builds a chunk with coherent grammar and syntax. By combining high-speed planning with precise local execution, CoDiLA generates high-quality text twice as fast as traditional models, paving the way for faster and more reliable AI assistants.