NeurIPS: Neuro-anatomical Inductive Priors for Sphere-based Brain Decoding

Current fMRI decoders face a performance-fidelity trade-off where efficient ID encoders outperform geometrically faithful surface-based models. We argue this is partly driven by inefficient surface tokenization and the failure to use anatomy as a predictive signal. We present NeurIPS, a framework that improves surface-based decoding by reframing anatomical variation from a nuisance to a powerful inductive prior. NeurIPS unites two innovations: a Selective ROI Spherical Tokenizer (SRST) for efficient geometric encoding, and a Structure-Guided Mixture of Experts (SG-MoE) that explicitly models individual anatomy using cortical features. On the Natural Scenes Dataset, NeurIPS establishes a new state-of-the-art for surface decoders and achieves performance comparable to strong 1D baselines. This is achieved with unprecedented efficiency, as the model converges dramatically faster (10 vs. 600 epochs). This efficiency enables rapid adaptation to new subjects using only 20\% of data and ensures robust scalability as the training cohort is expanded. Ablations provide causal evidence that these gains are driven by the model's use of cortical features, not by memorizing subject IDs. By leveraging anatomical priors, NeurIPS provides a principled and scalable path toward robust, generalizable brain decoding.

Brain imaging can help infer what a person is seeing from functional MRI (fMRI) signals, but current systems are difficult to use across different people because every brain has a slightly different shape and organization. Many strong fMRI-to-image methods simplify brain signals into one-dimensional vectors, which is efficient but loses the natural geometry of the cortex. Our work introduces NeurIPS, a brain decoding model that keeps this cortical geometry while making surface-based decoding more practical. It uses a Selective ROI Spherical Tokenizer to focus computation on visual brain regions, and a Structure-Guided Mixture of Experts to use anatomical features such as cortical thickness, curvature, and sulcal depth when adapting across people. This lets the model treat individual anatomy as useful information rather than noise. On the Natural Scenes Dataset, NeurIPS improves over prior surface-based brain decoders and comes close to strong one-dimensional methods. It also adapts quickly to a new subject with limited training data. These results suggest that brain anatomy can help build more scalable and reliable brain decoding systems, while also highlighting the need for careful safeguards around consent and privacy.