Diffusion Language Model Parallel Decoding via Product-of-Experts Bridge

Diffusion language models (DLMs) offer substantial speed advantages through parallel decoding, but the lack of token dependencies limits generation quality compared to autoregressive (AR) models. Recent progress attempts to bridge the gap via importance sampling, with DLM being the proposal and AR being the target. However, due to the huge gap between their distributions, the sampling requires a large number of particles and is thus expensive to compute. In this paper, we introduce PoE-Bridge, a novel decoding framework that drastically improves generation speed and accuracy by introducing an intermediate distribution to bridge the gap. The distribution is constructed as a Product-of-Experts (PoE) of the DLM proposal and the AR target. With the intermediate distribution, we first use the DLM to draft multiple continuations in parallel, then apply rejection sampling to verify the drafted tokens and move the resulting candidates toward the PoE. We then use importance sampling to further correct the PoE-aligned candidates toward the AR target. We further propose several improved techniques, including mixed-temperature sampling for enhanced diversity and elastic rejection windows for reducing wasted verification. Empirically, PoE-Bridge achieves significantly improved accuracy with $5\times$ speedup over the standard DLM decoding approach, and recovers at least 95% of the target AR model's performance, efficiently advancing most of the quality gap on challenging mathematical reasoning and coding tasks. Our code is available at [https://github.com/juntongshi48/poe-bridge](https://github.com/juntongshi48/poe-bridge).

Modern language models are powerful, but they generate text slowly because they produce one token at a time. This sequential process becomes a major bottleneck for applications that require long outputs, such as reasoning systems, coding agents, and reinforcement learning training. Diffusion language models offer a promising alternative because they can generate many tokens in parallel, potentially making text generation much faster. However, this speed advantage usually requires sacrificing quality. A natural solution is to let the fast diffusion model draft text first and then use a stronger standard language model to verify or correct the drafts. Unfortunately, the two models tend to produce very different kinds of text, so most drafted tokens end up being rejected. In practice, this often collapses back to generating nearly one token at a time. Our work solves this problem by introducing an intermediate “bridge” between the two models. Instead of forcing one difficult correction step, we break the process into smaller and easier refinements. Surprisingly, this preserves the parallel generation ability of diffusion models while still producing outputs very close in quality to strong standard language models. Across all math and coding tasks we tested, our method recovered at least 95% of the stronger model’s accuracy while generating text $2\times$ faster than the stronger model alone. Such a fast decoding algorithm is becoming increasingly important as AI systems grow more interactive and reasoning-intensive. Reinforcement learning systems must generate massive numbers of simulated interactions, while coding and reasoning agents often produce long chains of intermediate thoughts before arriving at a final answer. Methods that reduce generation latency can therefore make these systems significantly more efficient and responsive.