Emergent Symbolic Structure in Health Foundation Models: Extraction, Alignment, and Cross-Modal Transfer

Health foundation models (FMs) learn useful rep- resentations from wearable sensors, but interpret- ing what they encode and transferring that knowl- edge across modalities after training remains dif- ficult. We present a post-training framework that decomposes frozen embeddings into interpretable directions, referred to as symbols, and use these symbols to align the embedding spaces without re- training. We evaluate the framework on three FMs for photoplethysmography (PPG) and accelerom- eter data, independently pretrained on ∼20M min- utes of unlabeled data from ∼172K participants, and analyzed on a held-out cohort of 30K sub- jects. We find that extracted symbols associate selectively with health conditions and physiologi- cal attributes, and these associations are partially shared across modalities and architectures. Cross- modal transfer via symbols retains more than 95% of in-domain performance, is nearly symmetric across domain directions, and saturates with lim- ited paired data, together indicating that align- ment recovers a shared low-dimensional subspace rich in physiological information. Overall, these results suggest that health FM embeddings con- tain an interpretable symbolic organization that is shared across modalities and supports cross- domain transfer without joint training.