SafeSeek: Universal Attribution of Safety Circuits in Language Models

Mechanistic interpretability reveals that safety-critical behaviors (e.g., alignment, jailbreak, backdoor) in Large Language Models (LLMs) are grounded in specialized functional components. However, existing safety attribution methods struggle with generalization and reliability due to their reliance on heuristic, domain-specific metrics and search algorithms. To address this, we propose SafeSeek, a unified safety interpretability framework that identifies functionally complete safety circuits in LLMs via optimization. Unlike methods focusing on isolated heads or neurons, SafeSeek introduces differentiable binary masks to extract multi-granular circuits through gradient descent on safety datasets, while integrates Safety Circuit Tuning to utilize these sparse circuits for efficient safety fine-tuning. We validate SafeSeek in two key scenarios in LLM safety: \textbf{(1) backdoor attacks}, identifying a backdoor circuit with 0.42\% sparsity, whose ablation eradicates the Attack Success Rate (ASR) from 100\% $\to$ 0.4\% while retaining over 99\% general utility; \textbf{(2) safety alignment}, localizing an alignment circuit with 3.03\% heads and 0.79\% neurons, whose removal spikes ASR from 0.8\% $\to$ 96.9\%, whereas excluding this circuit during helpfulness fine-tuning maintains 96.5\% safety retention.

(1) Large language models (LLMs) can sometimes exhibit unsafe behaviors, such as being manipulated by hidden "backdoors" or failing to refuse harmful requests, often due to specific, sparse internal components known as "safety circuits." (2) We developed SafeSeek, a general framework that acts like "AI brain surgery" to precisely locate and isolate these critical circuits responsible for safe or unsafe actions. (3) By removing malicious circuits or reinforcing safety ones, our method can efficiently fix model vulnerabilities or enhance its safety, all while preserving nearly all of the model's original useful capabilities.