Physiology-Aware Masked Cross-Modal Reconstruction for Biosignal Representation Learning

Biosignals acquired from different locations on the body often provide temporally ordered views of the same underlying physiological process. However, most existing self-supervised learning methods treat these signals as interchangeable views, overlooking the directional temporal dynamics that link them. A canonical example is the relationship between electrocardiography (ECG), which captures the electrical activation initiating each heartbeat, and photoplethysmography (PPG), which records the resulting peripheral pulse delayed by vascular dynamics. To capture this structured relationship, we introduce xMAE, a biosignal pretraining framework that leverages masked cross-modal reconstruction across temporally ordered biosignals as a training-time constraint to encourage physiologically meaningful timing structure in the learned representations. We show that pretraining with xMAE yields representations that outperform both unimodal and multimodal baselines on 15 of 19 downstream tasks, including cardiovascular outcome prediction, abnormal laboratory test detection, sleep staging, and demographic inference, while generalizing across devices, body locations, and acquisition settings. Further analysis suggests that the ECG--PPG timing structure is reflected in the learned PPG representations. Code is available at https://github.com/hzhou3/xMAE.

Wearable devices such as smartwatches and rings can measure photoplethysmography (PPG), an optical signal related to blood flow. PPG is easy to collect passively and unobtrusively, but it captures the heart’s activity indirectly through peripheral blood flow, while electrocardiography (ECG) directly measures the heart’s electrical activity. In this work, we introduce xMAE, a multimodal learning framework that uses paired ECG and PPG signals during training to help a model learn better PPG representations. The key idea is that ECG happens first and PPG follows after a short physiological delay, so the model should learn this directional relationship rather than treating the two signals as interchangeable. After training, the model only needs PPG, making it suitable for wearable devices. Across multiple health-related tasks, including cardiovascular conditions, sleep staging, and blood biomarkers, xMAE learns representations that transfer better than many existing methods.