Learning Fingerprints for Medical Time Series with Redundancy-Constrained Information Maximization

Learning meaningful representations from medical time series (MedTS), such as ECG or EEG signals, is a critical challenge. These signals are often high-dimensional, variable-length, and rife with noise. Existing self-supervised approaches, such as Masked Autoencoders (MAEs), are highly effective for pre-training general-purpose encoders. However, they do not explicitly learn compact, fixed-size, or semantically interpretable latent representations, typically relying on heuristic aggregation strategies such as global average pooling or a designated [CLS] token. We propose a novel framework that compresses a variable-length MedTS into a fixed-size set of $k$ latent Fingerprint Tokens. Our architecture employs a cross-attention bottleneck to generate these tokens and is trained with a dual-objective function. The first objective is a reconstruction loss, which ensures the tokens are \textit{sufficient statistics} for the original data. The second, a diversity penalty based on the Total Coding Rate (TCR), explicitly minimizes the redundancy between tokens, encouraging them to become statistically \textit{disentangled} representations. We present the theoretical justification for our method, framing it as a novel \textbf{Disentangled Rate-Distortion} problem. This approach produces a low-dimensional, interpretable, and sample-efficient representation, where each token is encouraged to capture an independent factor of variation, paving the way for more robust digital biomarkers.

Medical devices often record long and noisy signals, such as heart activity from ECG or brain activity from EEG. These signals can help detect disease, but they are difficult for machine learning models to summarize in a way that is both accurate and understandable. Many existing methods compress the whole signal into one tangled representation, making it hard to know which parts of the signal matter for a prediction. We introduce TS-Fingerprint, a method that turns each medical time series into a small set of “fingerprint” tokens. Each token is encouraged to capture different information, so the model avoids repeatedly storing the same signal pattern. During training, the model learns to reconstruct the original signal while also keeping these tokens diverse. Across several ECG, EEG, and activity-recognition datasets, TS-Fingerprint improves classification performance compared with strong time-series and medical baselines. More importantly, the learned tokens can specialize in different signal patterns, making the model’s decisions easier to inspect. This may help researchers build more reliable and interpretable tools for analyzing physiological signals, while still leaving final clinical judgment to medical experts.