ESSA: Evolved Safety Specification Alignment

Safety alignment in large language models (LLMs) often suffers from miscalibration: models may over-refuse benign requests while remaining vulnerable to multi-turn jailbreaks that poison conversational context. We propose Evolved Safety Specification Alignment (ESSA), a post-training framework that teaches models to deliberate with explicit safety specifications conditioned on both domain and task. ESSA begins with a three-tier hierarchical constitution that sets global priorities and interpretation rules. Building on this foundation, we automatically evolve compact safety specification sets for each (domain, task) pair via an LLM-driven evolutionary loop with multi-perspective judging and redundancy control. We then synthesize a training dataset of deliberation traces and responses under the evolved specifications, and fine-tune a target model on it. Across safety suites, ESSA improves multiple safety metrics and is consistently preferred in rubric-based pairwise evaluation, while changes on general capability benchmarks remain mixed but modest.