Self-Supervised Dynamical System Representations for Physiological Time-Series

Self-supervised learning for physiological time-series aims to captures the identity of the underlying dynamical process while filtering irrelevant noise. However, existing approaches may obscure the clinical semantics important for downstream transferability. Weakly constrained pretext tasks (i.e. contrastive learning, MAE) may incorrectly ignore the underlying dynamical structure, while structurally constrained models (i.e. SVAEs) are unable to selectively filter sample-specific noise. To bridge this gap, we propose ${\bf PULSE}$, a novel pretraining objective that simultaneously preserves dynamical relationships important to physiological time-series while selectively removing irrelevant noise. We achieve this by formulating a dynamical systems model to identify transferable and non-transferable information between time-series windows, and target the former through a novel cross-reconstruction objective. We establish theory that provides conditions for when transferrable information is recovered, and empirically validate it through synthetic experiments. On several real-world datasets, PULSE effectively distinguishes clinical semantic classes, increases label efficiency, and improves transfer learning performance.

Teaching a machine learning model to focus on the meaningful patterns in physiological recordings like heart or brain activity, without hand-labeling huge amounts of data, remains difficult. These recordings vary widely from sample to sample, and useful patterns are often buried in noise. Existing methods tend to fail in one of two ways: some ignore the underlying biological dynamics that distinguish one signal from another, while others retain too much, unable to separate meaningful structure from irrelevant noise. We introduce PULSE, a method that learns to keep the information shared across recordings while discarding what is specific to a single sample. We show mathematically when this is possible and confirm it in controlled experiments. On real medical datasets, PULSE better distinguishes clinically meaningful categories, needs fewer labels, and transfers more effectively to new settings.