Toward Identifiable Sparse Autoencoders

Recently, sparse autoencoders (SAEs) have emerged as an attractive tool for interpreting and interacting with representations in practical neural networks. While it is common empirical folklore, we also show theoretically that SAEs are highly unstable: different training runs are likely to produce different concept dictionaries and sparse codes. We characterize the model properties that hinder the stability of real-world SAEs, and address each of these problems through minimal changes to the architecture and training procedure. Together, these changes yield two versions of an identifiable SAE (iSAE), a variant of the standard TopK SAE with lower reconstruction error and improved stability. We explain this improvement theoretically by connecting SAEs with traditional dictionary learning approaches, and show that the dictionaries learned in practice satisfy an approximate restricted isometry condition, rendering the corresponding sparse codes in those models near-identifiable.

How do large language models and other modern machine learning models arrive at their output? We usually don’t fully know. Modern neural networks store information in complicated internal representations that are difficult for humans to interpret. Sparse autoencoders (SAEs) are an interesting tool which aim to uncover the “hidden concepts” behind a model’s output by rewriting a model’s internal state using a smaller set of human-understandable features. However, this paper shows that standard SAEs are fundamentally unstable: training the same model multiple times often produces different “explanations” for the same behavior. We identify the causes of this instability and introduce a modified architecture, called the identifiable SAE (iSAE), that learns more stable and reliable concepts while also improving reconstruction quality. Finally, we connect these results to classical dictionary learning theory, showing mathematically why the learned representations in iSAEs can become much closer to uniquely identifiable.