Mechanisms of Introspective Awareness

Recent work shows that LLMs can sometimes detect when steering vectors are injected into their residual stream and identify the injected concept, a phenomenon cited as evidence of "introspective awareness." But what mechanisms underlie this capability, and do they reflect genuine introspective circuitry or more shallow heuristics? We investigate these questions in open-source models and establish three main findings. First, introspection is behaviorally robust: detection achieves moderate true positive rates with 0% false positives across diverse prompts. We also find this capability emerges specifically from post-training rather than pretraining. Second, introspection is not reducible to a single linear confound: anomaly detection relies on distributed MLP computation across multiple directions, implemented by interpretable gate and evidence-carrier features. Third, models possess greater introspective capability than is elicited by default: ablating refusal directions improves detection by ~50% and a trained steering vector improves detection by ~75%. Overall, our results suggest that introspective awareness is behaviorally robust, grounded in nontrivial internal anomaly detection, and likely could be substantially improved in future models.

Language models can sometimes tell when researchers deliberately modify their internal computations and can even identify what kind of information was added. We studied how this ability works and whether it reflects self-monitoring of the model’s own computations or simpler shortcuts. We find that this behavior is robust across different settings and appears to emerge during the post-training stage that turns pretrained models into chat assistants. Models use several internal components that work together to detect unusual activity. We also find that models can often detect these perturbations more accurately than their default responses reveal: targeted interventions such as ablating refusal behavior and training a small additive vector can substantially improve reporting. Overall, our results suggest that current language models have a limited but mechanistically grounded ability to recognize some changes to their own internal computations, and that this capability could become significantly stronger in future systems.