Fantastic Reasoning Behaviors and Where to Find Them: Unsupervised Discovery of the Reasoning Process

Despite the growing reasoning capabilities of recent large language models (LLMs), their internal mechanisms during the reasoning process remain underexplored. Prior approaches often rely on human-defined concepts (e.g., overthinking, reflection) at the word level to analyze reasoning in a supervised manner. However, such methods are limited, as it is infeasible to capture the full spectrum of potential reasoning behaviors, many of which are difficult to define in token space. In this work, we propose an unsupervised framework (namely, RISE: Reasoning behavior Interpretability via Sparse auto-Encoder) for discovering reasoning vectors, which we define as directions in the activation space that encode distinct reasoning behaviors. By segmenting chain-of-thought traces into sentence-level 'steps' and training sparse auto-encoders (SAEs) on step-level activations, we uncover disentangled features corresponding to interpretable behaviors such as reflection and backtracking. Visualization and clustering analyses show that these behaviors occupy separable regions in the decoder column space. Moreover, targeted interventions on SAE-derived vectors can controllably amplify or suppress specific reasoning behaviors, altering inference trajectories without retraining. Beyond behavior-specific disentanglement, SAEs capture structural properties such as response length, revealing clusters of long versus short reasoning traces. More interestingly, SAEs enable the discovery of novel behaviors beyond human supervision. We demonstrate the ability to control response confidence by identifying confidence-related vectors in the SAE decoder space. These findings underscore the potential of unsupervised latent discovery for both interpreting and controllably steering reasoning in LLMs.

Modern AI reasoning models like DeepSeek and Qwen can solve hard math problems by "thinking out loud" — writing long chains of reasoning before answering. But we don't really understand what's happening inside these models as they think. Prior attempts to study this relied on humans manually defining behaviors to look for, which misses anything we didn't think to label in advance. We built RISE, an unsupervised framework that automatically discovers how reasoning behaviors are organized inside a model's internal representations — no human labels needed. By training a sparse autoencoder on the model's activations at each reasoning step, RISE finds geometric directions in the model's internal space that correspond to distinct behaviors like reflection (double-checking prior steps) and backtracking (abandoning an approach and trying another). We can then steer these directions at inference time to amplify or suppress specific behaviors without any retraining. RISE also uncovered a previously hard-to-define behavior — response confidence — and showed that nudging it reduces unnecessary reflection while maintaining accuracy. This opens a scalable path to understanding and controlling how large reasoning models reason, potentially enabling more efficient, reliable, and interpretable reasoning systems.