Mind the State: Towards Unified, Context-Aware EEG-to-fMRI Synthesis

Functional magnetic resonance imaging (fMRI) provides dynamic measurements of human brain activity at high spatial resolution and depth, but its use is constrained by high cost, limited accessibility, and strict acquisition requirements. Synthesizing fMRI data from more accessible, non-invasive modalities such as electroencephalography (EEG) offers a promising alternative, enabling inference of deep brain dynamics from low-cost scalp recordings in naturalistic settings. Despite recent progress, existing EEG-to-fMRI translation methods typically rely on region-specific models and offer limited support for subject-level and dataset-level heterogeneity, restricting their generalizability. We propose UniEFS, a unified EEG-to-fMRI Synthesis model that enables full-brain fMRI reconstruction while accommodating varying demographic and physiological contexts within a single model. Our approach leverages a pretrained fMRI decoder to embed rich spatial priors and introduces condition-aware prompt tokens that encode subject-level and experimental metadata, enabling effective handling of heterogeneous datasets. We extensively evaluate the model performance on eyes-closed resting-state data and demonstrate that it can reliably reconstruct temporally-resolved whole-brain fMRI activity, with potential to generalize to task-based fMRI and clinical populations in a zero-shot manner. Project page: https://soupeeli.github.io/UniEFS

Brain imaging is important for understanding how the human brain works and for studying brain-related disorders. Functional MRI can measure activity across the whole brain with high spatial detail, including deep brain areas, but it is expensive, difficult to access, and requires specialized equipment. EEG, on the other hand, is much cheaper, portable, and easier to use, but it provides much less information about where activity occurs inside the brain. In this paper, we introduce UniEFS, a machine learning model that aims to bridge this gap by estimating fMRI-like brain activity from EEG recordings. Instead of building separate models for different brain regions, UniEFS uses one unified model to predict activity across the whole brain. It also takes into account information such as participant physiological and demographic characteristics, as well as experimental metadata, which can affect the relationship between EEG and fMRI signals. We evaluate UniEFS on resting-state brain recordings and show that it can reconstruct time-varying whole-brain fMRI activity from EEG. We also test whether the model can generalize to new settings, including task-based recordings and EEG data from clinical populations where fMRI is not available. These results suggest that EEG may one day provide more detailed information about brain activity than is currently possible, potentially supporting more accessible brain monitoring in research and healthcare settings. Further validation on larger and more diverse datasets will be needed before clinical use.