Harmfulness Propagation Dynamics: Layer-wise Trajectories of Adversarial Intent in Large Language Models
Noor I Mohammad
Abstract
We identify Harmfulness Propagation Dynamics (HPD). For harmful prompts, the projection of the last-token hidden state onto a learned harm direction rises monotonically with transformer depth, whereas benign prompts remain flat or oscillatory. This cross-layer signature reflects harmful intent as a progressively resolved semantic property: surface form appears early, while pragmatic intent consolidates later, making the trajectory shape more informative than any single-layer snapshot. Moreover, LDA-based harm directions, learned per layer, remain stable across random splits (pairwise cosine similarity $>0.97$), supporting the projection sequence as a reproducible structured signal. Building on HPD, we introduce HERALD (Harmful Encoding Recognition via Activation Layer Dynamics). This lightweight input moderator extracts a seven-dimensional feature record, slope, curvature, monotonicity, onset layer, and related statistics from the cross-layer projection sequence and classifies it with a 288-parameter MLP. \herald{} stores one $d$-dimensional direction per layer ($262$\,KB for a 32-layer, $d{=}4096$ model), requires no gradient computation during training, and adds only $2.6{\times}10^{-6}$ prefill FLOPs at inference. Across eight prompt-harmfulness benchmarks and four model families, HERALD achieves an average F1 score of $89.3$ on OLMo2-7B, surpassing all tested guard models on adversarial jailbreak detection ($98.4$ vs. $96.9$ F1) and outperforming prior latent-based methods by $2.3$-$4.1$ F1 points across all backbones. Per-instance trajectories provide machine-readable audit records that reveal when and how harmfulness emerges, offering an interpretability advantage over single-layer approaches. Code will be released at https://HERALD.github.io.
Chat is not available.
Successful Page Load