Toward Stable Value Alignment: Introducing Independent Modules for Consistent Value Guidance

Aligning large language models (LLMs) with human values typically relies on post-training or inference-time steering that directly manipulates the backbone’s parameters or representation space. However, a critical gap exists: the model’s residual stream is highly dynamic, in which values exist as fragile, low-dimensional properties, inherently incompatible with the stability required for consistent value expression. In this paper, we propose the Stable Value Guidance Transformer (SVGT), which addresses this gap through an independent value module incorporating two key designs: (1) independent value modeling, maintaining normative representations in a dedicated value space isolated from the backbone, and (2) explicit behavioral guidance, transducing these stable signals into learnable latent Bridge Tokens. These tokens serve as dynamic value anchors to explicitly steer the generative trajectory, ensuring robust adherence across diverse contexts without disrupting the backbone’s internal representations. Experiments across multiple backbones and safety benchmarks show that SVGT generally reduces harmful scores by over 70\% while maintaining generation fluency, demonstrating the efficacy of architecturally grounded value modeling.

Large language models (LLMs) are increasingly used in situations where their responses need to be not only fluent, but also safe and consistent with human values. Yet these models can behave inconsistently: a safety-related behavior that appears stable in one context may weaken or disappear in another, especially when the prompt strongly pushes the model toward a risky response. This paper studies how to make value alignment more stable during generation. We introduce the Stable Value Guidance Transformer (SVGT), a method that keeps value-related reasoning in a separate module rather than mixing it directly into the main language model. This module observes the model’s internal state, evaluates whether the current generation is drifting toward unaligned behavior, and provides guidance that steers the response back toward a safer direction. Across several language models and safety benchmarks, SVGT substantially reduces harmful generations while largely preserving fluency. The results suggest that separating value guidance from ordinary language generation can be a practical way to make language models more reliable across different contexts