Learning Self-Interpretation from Interpretability Artifacts: Training Lightweight Adapters on Vector-Label Pairs

Self-interpretation methods prompt language models to describe their own internal states, but remain unreliable due to hyperparameter sensitivity. We show that training lightweight adapters on interpretability artifacts, while keeping the LM entirely frozen, yields reliable self-interpretation across tasks and model families. A scalar affine adapter with just $d_\text{model}+1$ parameters suffices: trained adapters generate sparse autoencoder feature labels that outperform the training labels themselves (70% vs 50% generation scoring at 70B scale), identify topics with 94% recall@1 versus 1% for untrained baselines, and decode bridge entities in multi-hop reasoning that appear in neither prompt nor response, surfacing implicit reasoning without chain-of-thought. The learned bias vector alone accounts for 85% of improvement, and simpler adapters generalize better than more expressive alternatives. Controlling for model knowledge via prompted descriptions, we find self-interpretation gains outpace capability gains from 7B to 72B parameters. Our results demonstrate that self-interpretation improves with scale, without modifying the model being interpreted.

Modern AI language models do their "thinking" in huge arrays of internal numbers that people can't read directly. One appealing idea is to feed one of these activation patterns into the model in place of a word and ask the model to explain it. This works, but unpredictably: the explanations sometimes sound sensible yet are entirely made up. We show that a tiny add-on, an "adapter" with only a few thousand numbers, fixes this. We train it on examples that pair an internal pattern with a known description of its meaning, data the field already produces in bulk. Crucially, the add-on feeds these patterns back into the same model, so the model interprets itself. This opens the door to future AI that can examine its own thinking from the inside. The model then reliably describes what its hidden states represent, sometimes more accurately than the descriptions used to train the add-on itself. Strikingly, when a model answers a multi-step question instantly with no visible reasoning, our method can often recover the intermediate fact it worked out silently but never stated. Surfacing such hidden steps could help us verify whether AI systems are being honest about what they're really doing.