Metis: Learning to Jailbreak LLMs via Self-Evolving Metacognitive Policy Optimization

Red teaming is critical for uncovering vulnerabilities in Large Language Models (LLMs). While automated methods have improved scalability, existing approaches often rely on static heuristics or stochastic search, rendering them brittle against advanced safety alignment. To address this, we introduce \textbf{Metis}, a framework that reformulates jailbreaking as inference-time policy optimization within an adversarial Partially Observable Markov Decision Process (POMDP). Metis employs a self-evolving metacognitive loop to perform causal diagnosis of a target's defense logic and leverages structured feedback as a semantic gradient to refine its policy, offering enhanced interpretability through transparent reasoning traces. Extensive evaluations across 10 diverse models demonstrate that Metis achieves the strongest average Attack Success Rate (ASR) among compared methods at 89.2\%, maintaining high efficacy on resilient frontier models (e.g., 76.0\% on O1 and 78.0\% on GPT-5-chat) where traditional baselines exhibit substantial performance degradation. By replacing redundant exploration with directed optimization, Metis reduces token costs by an average of 8.2$\times$ (and up to 11.4$\times$). Our analysis reveals that current defenses remain vulnerable to internally-steered, closed-loop reasoning trajectories under the tested settings, highlighting a critical need for next-generation defenses capable of reasoning about safety dynamically during inference.

Large language models are increasingly used in everyday tools, but they can still be manipulated into producing unsafe responses. Safety teams need reliable ways to find these weaknesses before attackers do, especially in multi-turn conversations where a harmful request may emerge gradually rather than in a single prompt. This paper introduces Metis, an automated red-teaming system that tests language models through an adaptive conversation. Instead of repeatedly trying fixed attack templates, Metis observes how a target model responds, reasons about what kind of safety boundary blocked the previous attempt, and adjusts its next strategy accordingly. The system also records its reasoning process, so researchers can inspect not only whether an attack succeeded, but why it succeeded or failed. Across a broad set of language models and benchmarks, Metis finds vulnerabilities more effectively and with lower token cost than the compared automated red-teaming methods. More importantly, its recorded trajectories can help safety researchers study failure patterns, build stronger defenses, and provide process-level supervision for improving multi-turn safety alignment.