ROAMM: A Benchmark Dataset for Multimodal Human Attention Decoding and EEG-to-Text Modeling During Naturalistic Reading

We present Reading Observed At Mindless Moments (ROAMM), a multimodal dataset comprising 50 hours of simultaneous EEG and eye-tracking recordings collected during naturalistic multi-page reading from 44 participants. ROAMM includes synchronized physiological recordings, eye-movement events, page-level comprehension scores, and span-level mind-wandering (MW) annotations obtained using a retrospective self-report paradigm. We introduce a standardized leave-one-subject-out benchmark for MW detection and achieve up to 0.609 AUROC using supervised models. We additionally evaluate EEG-to-text decoding on reading segments with and without MW labels, showing that decoding performance decreases during MW episodes. ROAMM enables research on MW detection, EEG-to-text decoding, multimodal representation learning, and attention-related degradation of language representations during naturalistic reading.

Our minds often drift away from a task without us noticing. These lapses are called mind-wandering, or task-unrelated thoughts. They occur across many everyday activities but are often overlooked during human data collection. This raises an important question: do these natural fluctuations in attention affect the AI models trained on such datasets? To answer this, we introduce ROAMM, a dataset that records both brain activity (EEG) and eye movements while people read full articles in a natural setting, with span-level mind-wandering annotations from our ReMind task paradigm. Using this data, we train models to detect when a reader is focused and to decode text from brain signals during both attentive and mind-wandering periods. We find that combining brain and eye signals improves detection of attention lapses, and that mind-wandering significantly reduces how accurately text can be decoded from brain activity. ROAMM contributes to a small but growing set of datasets that enable the machine learning community to study neural decoding in realistic settings. More importantly, it supports the development of more reliable, attention-aware AI systems, with potential applications in brain–computer interfaces and our broader understanding of human cognition.