Fast and Accurate Causal Parallel Decoding using Jacobi Forcing

Multi-token generation has emerged as a promising paradigm for accelerating language model inference, with the diffusion Large Language Models (dLLMs) as the most notable approach recently. Popular dLLMs like SDAR and Fast-dLLM v2 are post-trained on pre-trained AR models to minimize training cost while maintaining high performance. However, there exists a fundamental pretrain-to-posttrain mismatch -- the masked data distribution and bidirectional attention in post-training deviates significantly from the real data distribution and causal attention for pretraining. As a result, the post-trained dLLMs usually suffer from limited speedup or substantially degraded performance. To address this, we introduce Jacobi Forcing to bypass the dLLM formulation, directly post-training a causal multi-token predictor from an AR LLM. In particular, we force the model to learn to leap along its own parallel token generation trajectories based on Jacobi Decoding, and introduce an elaborate progressive distillation paradigm. The trained models achieve $3.8\times$ wall-clock speedup on coding and math benchmarks with minimal loss in performance. Based on the trajectory characteristics of the model, we further introduce multi-block decoding with rejection recycling, which enables up to $4.6\times$ higher token acceptance count per iteration and $4.0\times$ wall-clock speedup, effectively trading additional compute for lower inference latency.

Large language models are powerful, but they write text one token at a time, which makes them slow for long answers, coding, math, and interactive AI applications. A natural way to speed them up is to make them predict many future tokens at once, but existing approaches often lose quality since they require training methods that do not match how standard language models were originally built. We introduce Jacobi Forcing, a training method that teaches an existing autoregressive language model to become a fast parallel decoder while keeping its left-to-right reasoning structure. Instead of asking the model to denoise masked text like a diffusion model, we train it on its own parallel generation trajectories, gradually making the task harder so it learns to recover correct future tokens even from noisy intermediate drafts. This lets the model generate multiple useful tokens in a single step, substantially reducing the number of model calls needed to produce an answer. We also introduce decoding techniques that reuse good partial drafts and refine multiple blocks of text at the same time. Together, these methods achieve more than 4 times faster generation on coding and math benchmarks while largely preserving accuracy, making high-quality language models more practical for latency-sensitive applications.