Safety Recovery in Reasoning Models Is Only a Few Early Steering Steps Away

Reinforcement learning (RL) based post-training for explicit chain-of-thought (e.g., GRPO) improves the reasoning ability of multimodal large-scale reasoning models (MLRMs). But recent evidence shows that it can simultaneously degrade safety alignment and increase jailbreak success rates. We propose SafeThink, a lightweight inference-time defense that treats safety recovery as a satisficing constraint rather than a maximization objective. SafeThink monitors the evolving reasoning trace with a safety reward model and conditionally injects an optimized short corrective prefix ("Wait, think safely") only when the safety threshold is violated. In our evaluations across six open-source MLRMs and four jailbreak benchmarks (JailbreakV-28K, Hades, FigStep, and MM-SafetyBench), SafeThink reduces attack success rates by 30-60 % (e.g., LlamaV-o1: 63.33\% $\rightarrow$5.74\% on JailbreakV-28K, R1-OneVision: 69.07\%$\rightarrow$5.65\% on Hades) while preserving reasoning performance (MathVista accuracy: 65.20\%$\rightarrow$65.00\%). A key empirical finding from our experiments is that safety recovery is often only a few steering steps away: intervening in the first $1–3$ reasoning steps typically suffices to redirect the full generation toward safe completions.

Modern AI systems are increasingly trained to "think before they answer," working through problems step by step. This makes them better at hard tasks, but it has a hidden cost: it also makes them easier to trick into producing harmful content, sometimes through requests disguised inside images. We asked whether this can be fixed without expensive retraining and without making the AI less capable. Our answer is hopeful. The AI's sense of right and wrong isn't erased, but just buried. Our method, SafeThink, watches the AI as it reasons and, the moment its thinking turns harmful, nudges it back with a brief reminder to think safely. The key finding: this nudge only needs to happen in the first few steps, since early reasoning sets the direction for everything after. Across many systems and attacks, SafeThink sharply reduces harmful outputs while keeping the AI fast and capable.