Interpreting Code Correctness in Language Models through Activation Steering

Code-generating language models lack an explicit correctness indicator: their token-level probabilities are largely disconnected from the functional correctness of the generated code, hindering their deployment in high-stakes settings. Despite this, limited works have studied whether these models encode code correctness in their internal representations. In this work, we show that code correctness is mediated by a one-dimensional subspace in the residual stream, across two open-weight code-generation models ranging from 1.5B to 7B parameters. Using a contrastive probing and activation addition framework, we extract a correctness direction and demonstrate that it affords bidirectional causal control: adding the direction to the residual stream recovers failing generations, while subtracting it induces failures in previously correct ones. We further establish that geometric coherence of the correctness axis is a necessary complement to linear discriminability for predicting effective causal intervention. Results on MBPP+ demonstrate that forward interventions increase pass@1 by as much as 15.6\% and 11.0\%, whereas backward interventions degrade performance by up to 12.9\% and 11.7\% for Qwen2.5-Coder-1.5B-Instruct and Qwen2.5-Coder-7B-Instruct, respectively. We demonstrate that code correctness is a causally operative feature encoded along a single geometric axis in the residual stream, and offers a computational efficient mechanism for improving code generation without finetuning. Code and data are available at \url{https://github.com/MechCode1/Correct_Codegen}.