Internalizing Safety Understanding in Large Reasoning Models via Verification

While explicit Chain-of-Thought (CoT) empowers large reasoning models (LRMs), it enables the generation of riskier final answers. Current alignment paradigms primarily rely on externally enforced compliance, optimizing models to detect malicious prompts rather than evaluating the safety of their own outputs. We argue that this approach remains largely behavioral: our empirical analysis reveals that ostensibly aligned models lack intrinsic safety understanding, often failing to verify their own response safety and remaining vulnerable to adversarial jailbreaks. To address this fundamental limitation, we propose Safety Internal (SInternal), a framework that internalizes safety specifications by training LRMs exclusively on safety verification tasks to critique their own generated answers using expert reasoning trajectories. We demonstrate that learning to verify induces a strong generalization for response safety, significantly enhancing robustness against out-of-domain jailbreaks. Furthermore, when combined with reinforcement learning, SInternal serves as a superior initialization compared to standard supervised fine-tuning, suggesting that internalizing safety understanding creates a more robust foundation for alignment than merely mimicking safe behaviors. Our codes are available at \url{https://github.com/AlphaLab-USTC/SInternal}

AI systems that reason step by step can solve difficult tasks, but the same reasoning can also make unsafe answers harder to prevent. Many current safety methods mainly teach these systems to recognize harmful requests or produce safe-looking refusals. However, our study finds that a model may still fail to judge whether its own answer is safe, especially when a harmful request is disguised or manipulates its reasoning. We propose Safety Internal (SInternal), a training method that teaches a model to review its own generated answers and explain whether they follow safety rules. Instead of only learning what a safe final response looks like, the model learns why some responses are unsafe. Experiments show that this makes models more robust to new and disguised harmful prompts. SInternal also provides a better starting point for later safety training than ordinary training on safe examples. Our results suggest that safer AI systems should learn to understand and check safety principles, not just imitate safe behavior.