On the Relationship Between Activation Outliers and Feature Death in Sparse Autoencoders

Sparse autoencoders (SAEs) decompose neural network activations into interpretable features, but many learned features never activate, a problem called feature death that wastes dictionary capacity and can reintroduce superposition. Death rates vary dramatically between models: near-zero on GPT-2, over 70\% on AlphaFold3 with identical configurations. We find that dimension-level activation outliers (dimensions whose mean magnitude is large relative to per-token variation) cause this by shifting pre-activations at initialization based on each feature's alignment with the activation mean. Features anti-aligned with the mean receive permanently negative pre-activations and never fire. We formalize outlier severity as $\gamma = \|\boldsymbol{\mu}\|/\|\boldsymbol{\sigma}\|$; it predicts initial death rates (Spearman $\rho = 0.89$ for dead-by-TopK, $0.82$ for dead-by-ReLU) across 454 model-layer combinations spanning language, vision, protein, and genomic models. Dead features can revive during training, but recovery requires the SAE bias to learn the activation mean, a process that is prohibitively slow at high $\gamma$. Mean-centering (subtracting the activation mean) sidesteps this and eliminates outlier-induced death across all tested models, confirming the mechanism and providing a principled basis for when and why this preprocessing step is necessary.

Sparse autoencoders (SAEs) decompose the internal representations of neural networks into individual concepts that humans can examine. When SAEs are applied to many networks, especially in biology and vision, 50 to 95 percent of the concepts they try to learn end up unused, a problem called feature death. We trace this to a structural property of the activations: a small number of internal dimensions take consistently large values across all inputs, biasing the math so most of the SAE's concepts are locked into permanent inactivity before training even starts. A single quantity that can be measured before training captures how strong this bias is and predicts death rates across hundreds of layers spanning language, vision, protein, and genomic models. A simple fix, subtracting the average activation before training, removes the bias and eliminates death across every model we tested. Beyond providing a fix, the work clarifies why feature death affects some networks and not others, how the problem interacts with training dynamics, and when different preprocessing approaches succeed or fail, helping make SAEs more reliable for the diverse neural networks now being used in scientific applications like protein design.