SAEs-BrainMap: Unveiling the Emergence of Specialized Concepts in Deep Models via Brain Alignment

Understanding the internal mechanisms of deep neural networks remains a significant challenge, particularly in elucidating how generic visual concepts emerge within latent spaces. In this work, we propose SAEs-BrainMap, a novel framework that utilizes human brain activation patterns from the ventral visual pathway as objective probes to guide the identification of features decomposed by Sparse Autoencoders (SAEs). Our quantitative and qualitative empirical results demonstrate a robust representational alignment between sparse model features and biological Regions of Interests (ROIs), confirming the feasibility of utilizing brain signals to characterize model functionality. By leveraging this alignment, we trace the hierarchical trajectory of generic concepts cross layers and utilize the brain's hierarchical structure to visualize the model's global processing flow, providing novel insights into model interpretability. Our code is available at this repository.

Deep vision models can recognize images very well, but it is still difficult to understand what they learn inside and how their visual understanding develops across layers. In this work, we study this problem by using human brain activity as a guide. Specifically, we compare features inside vision models with activity patterns from the human visual cortex, the part of the brain involved in recognizing visual information. To make the model easier to analyze, we first break its internal representations into more interpretable components using sparse autoencoders. We find that some of these model components align well with different brain regions, suggesting that brain signals can help reveal what functions certain model features may represent. This also allows us to trace how visual concepts evolve from lower-level patterns to higher-level meanings across model layers. Our framework provides a brain-inspired way to better understand and visualize how deep vision models process images.