DC-Leap: Training-Free Acceleration of dLLMs via Draft-Guided Contiguous Leaping Decoding
Abstract
Lay Summary
Parallel decoding strategies for Diffusion Large Language Models (dLLMs) speed up generation by predicting multiple tokens simultaneously. However, to mitigate potential errors, these strategies use strict confidence thresholds, which force them to discard many potentially correct tokens, thereby bottlenecking the efficiency. Our work introduces DC-Leap, a training-free strategy to accelerate parallel decoding. We identify that much of the inefficiency stems from failing to properly account for the relationship between tokens decoded in a single forward process. DC-Leap addresses this by enforcing a sequential verification process that validates tokens within a dynamic window. Additionally, we use a draft-guided mechanism that allows the model to be aware of the future context, ensuring more stable predictions. By implementing these changes, DC-Leap increases the generation speed of dLLMs significantly across three models and five benchmarks without compromising accuracy. This makes dLLMs significantly more efficient for practical, real-world use.