Do Sparse Autoencoders Identify Reasoning Features in Language Models?

We study how reliably sparse autoencoders (SAEs) support claims about reasoning-related internal features in large language models. We first give a stylized analysis showing that sparsity-regularized decoding can preferentially retain stable low-dimensional correlates while suppressing high-dimensional within-behavior variation, motivating the possibility that contrastively selected "reasoning" features may concentrate on cue-like structure when such cues are coupled with reasoning traces. Building on this perspective, we propose a falsification-based evaluation framework that combines causal token injection with LLM-guided counterexample construction. Across 22 configurations spanning multiple model families, layers, and reasoning datasets, we find that many contrastively selected candidates are highly sensitive to token-level interventions, with 45%–90% activating after injecting only a few associated tokens into non-reasoning text. For the remaining context-dependent candidates, LLM-guided falsification produces targeted non-reasoning inputs that trigger activation and meaning-preserving paraphrases of top-activating reasoning traces that suppress it. A small steering study yields minimal changes on the evaluated benchmarks. Overall, our results suggest that, in the settings we study, sparse decompositions can favor low-dimensional correlates that co-occur with reasoning, underscoring the need for falsification when attributing high-level behaviors to individual SAE features. Code is available at https://github.com/GeorgeMLP/reasoning-probing.

Large language models often solve hard problems by writing out intermediate steps, but it is still unclear what internal mechanisms support this kind of reasoning. Our paper studies a popular interpretability tool called a sparse autoencoder, which is often used to find internal patterns that seem related to reasoning by comparing reasoning-style text with ordinary text. We show that this approach can be misleading, because the model may be reacting to simple words or writing styles that often appear in reasoning traces, rather than to the reasoning process itself. To test this, we combine theory with experiments that insert trigger words into non-reasoning text and use another language model to generate carefully designed counterexamples. Across many models, layers, and datasets, we find that most candidate “reasoning features” can be explained by these shallow cues. We also find that changing such features has little effect on benchmark performance. Our results suggest that researchers should be cautious when claiming that a single internal feature represents reasoning, and should use stronger causal tests before making such interpretations.