Transferable Lesion-Supervised Speech Representations for Post-Stroke Modelling

Characterising post-stroke brain injury typically relies on structural neuroimaging, which is costly, infrastructure-dependent, and poorly suited to repeated or large-scale monitoring; this also creates a data-efficiency problem, because supervised targets in clinical cohorts are often sparsely observed across modalities, limiting the amount of paired speech and clinical data available for downstream modelling. We investigate a transfer learning approach in which representations learned through lesion inference from speech are reused for downstream structural and clinical phenotypes absent from the original training signal. Specifically, patient-level aggregations of out-of-fold predicted lesion probabilities, stacked learner outputs, and uncertainty estimates are derived from a multi-head speech-to-lesion (S2L) model and benchmarked against clinically interpretable speech features, Whisper embeddings, clinical covariates, and demographic baselines under nested cross-validation. S2L representations emerge as the strongest direct predictors of focal lesion bounding-box extent ($R^2=0.241$, $r=0.503$), despite spatial extent being unseen during S2L training, while high-dimensional Whisper encoder embeddings perform best for cognitive outcomes ($R^2=0.486$, $r=0.702$), with further improvement after residual adaptation using S2L representations ($R^2=0.501$). Together, the findings suggest that transferred speech representations can help model post-stroke brain-behaviour relationships when clinical annotations are sparse, supporting more accessible brain-health monitoring.