Empirical Analysis of Layer Redundancy in Diffusion Language Models

Diffusion language models (DLMs) are promising alternatives to autoregressive models due to their bidirectional attention and parallel decoding. However, their inference cost becomes significantly higher as they scale. To address this challenge, we propose a dynamic layer-skipping framework. Our approach places a lightweight router before each Transformer layer to make unified, sequence-level execution decisions by aggregating masked token representations. Evaluated on LLaDA-8B across six benchmarks, we achieve a better FLOPs-accuracy trade-off than static and random baselines, including a 14.26\% FLOPs reduction on PIQA without accuracy loss. Furthermore, analysis reveals that initial layers are consistently critical, and layer redundancy naturally increases as the denoising process progresses.