ViEEG: Hierarchical Visual Neural Representation for EEG Brain Decoding

Understanding and decoding brain activity into visual representations is a fundamental challenge at the intersection of neuroscience and artificial intelligence. While electroencephalogram (EEG) visual decoding has shown promise due to its non-invasive and low-cost nature, existing methods suffer from {Hierarchical Neural Encoding Neglect (HNEN)}, a critical limitation in which flat neural representations fail to model the brain’s hierarchical visual processing. Inspired by the hierarchical organization of visual cortex, we propose ViEEG, a neuro-inspired framework that addresses HNEN. ViEEG decomposes each visual stimulus into three biologically aligned components, namely contour, foreground object, and contextual scene, which serve as anchors for a three-stream EEG encoder. These EEG features are progressively integrated via cross-attention routing, simulating cortical information flow from low-level to high-level vision. We further adopt hierarchical contrastive learning for EEG-CLIP representation alignment, enabling zero-shot object recognition. Extensive experiments on THINGS-EEG dataset demonstrate that ViEEG significantly outperforms previous methods by a large margin in both subject-dependent and subject-independent settings. Results on THINGS-MEG dataset further confirm ViEEG's generalization to different neural modalities. ViEEG not only advances the performance frontier but also sets a new paradigm for EEG brain visual decoding. Our code is available at https://github.com/LauMason/ViEEG.

Understanding what a person is seeing from brain activity is an important challenge in neuroscience and artificial intelligence, with potential applications in brain–computer interfaces and assistive technologies. In this work, we study how to recognize visual information from electroencephalography (EEG), a non-invasive technique that records brain signals from the scalp. Inspired by how the human visual system processes information in stages, we design a method that separates visual understanding into three levels: basic shapes and edges, foreground objects, and surrounding scene information. Our model learns to associate EEG signals with these different types of visual information and combines them to better understand what a person is viewing. We evaluate our method on widely used public EEG and MEG benchmarks and find that it improves visual recognition performance compared to previous approaches. Our results suggest that incorporating ideas from human visual processing may help build more effective brain decoding systems and contribute to future brain–computer interaction technologies.