Step-Wise Refusal Dynamics in Autoregressive and Diffusion Language Models
Eliron Rahimi ⋅ Elad Hirshel ⋅ Rom Himelstein ⋅ Amit LeVi ⋅ Avi Mendelson ⋅ Chaim Baskin
Abstract
Diffusion language models (DLMs) have recently emerged as a competitive alternative to autoregressive (AR) models, offering parallel decoding and controllable sampling dynamics while achieving competitive generation quality at scale. Despite this progress, the role of sampling mechanisms in shaping refusal behavior and jailbreak robustness remains poorly understood. In this work, we present an empirical study of step-wise refusal dynamics, examining the role of AR and diffusion sampling from a safety perspective. Our results strongly indicate that the sampling strategy (diffusion vs.\ AR) plays a central role in safety behavior, acting as a factor distinct from the underlying learned representations. To go beyond text-level analysis and provide interpretability, we introduce the Step-Wise Refusal Internal Dynamics (SRI) signal, which enables the analysis of safety failures (harmful generations), including cases of \emph{incomplete internal recovery} that are not observable at the text level. We further show that SRI leads to improved safety by enabling the construction of an inference-time jailbreak detector that generalizes to unseen attacks and achieves competitive state-of-the-art detection performance, while requiring over $100\times$ lower inference overhead compared to existing defenses.
Chat is not available.
Successful Page Load