Early Decisions Matter: Proximity Bias and Initial Trajectory Shaping in Non-Autoregressive Diffusion Language Models

Diffusion-based language models(dLLMs) have emerged as a promising alternative to autoregressive language models, offering the potential for parallel token generation and bidirectional context modeling. However, harnessing this flexibility for fully non-autoregressive decoding remains an open question, particularly for reasoning and planning tasks. In this work, we investigate non-autoregressive decoding in dLLMs by systematically analyzing its inference dynamics along the temporal axis. Specifically, we uncover an inherent failure modes in confidence-based non-autoregressive generation stem from a strong proximity bias—the denoising order tends to concentrate on spatially adjacent tokens. This local dependency leads to spatial error propagation, rendering the entire trajectory critically contingent on the initial unmasking position. Leveraging this insight, we present a minimal-intervention approach that guides early token selection, employing a lightweight planner and end-of-sequence temperature annealing. We thoroughly evaluate our method on various reasoning and planning tasks and observe substantial overall improvement over existing heuristic baselines without significant computational overhead.

Diffusion-based language models are a new type of language model that can generate multiple words at the same time, unlike traditional models that generate text one word after another. This makes them potentially faster and more flexible. However, it is still unclear how to make these models work reliably for complex tasks such as reasoning and planning. In this work, we study why fully parallel text generation often fails in diffusion language models. We find that the model tends to generate nearby words together, which can cause early mistakes to spread throughout the entire sentence. As a result, the quality of the output depends heavily on which parts of the text are generated at the initial stages. Based on this finding, we propose a simple method that guides the model to make better early decisions during generation. Our approach adds only minimal computational cost, but significantly improves performance on a variety of reasoning and planning tasks compared to existing methods.