Benchmarking Multimodal Clinical Foundation Models to Reveal Significant Demographic Disparities

Medical imaging foundation models produce high-dimensional structured feature vectors for downstream tabular classifiers, bridging visual pretraining and structured clinical data pipelines. We benchmark MedImageInsight, MedSigLIP, and BiomedCLIP on INSPECT, pairing CTPA imaging with longitudinal EHR, using linear and MLP probes across PE diagnostic and seven prognostic tasks. We report three findings. First, logistic regression outperforms all MLP variants for MedImageInsight (+0.014) and MedSigLIP (+0.021), indicating linearly separable structure in high-capacity embeddings. Second, structured EHR adds at most +0.003 AUROC over frozen CT alone (+0.017 for BiomedCLIP readmission), confirming fusion cannot compensate for representational deficits. Third and primarily, age is the dominant disparity: patients aged 18–40 face underdiagnosis rates (UDR) of 0.63–0.80 versus 0.31–0.41 for ages 75–90 (gap 0.32–0.45), exceeding race/ethnicity and gender gaps across all eight tasks. Adversarial debiasing is the only mitigation that reduces gaps without hurting AUROC: age-targeted debiasing cuts MedImageInsight’s gap by 79% (0.333→0.069; p<0.001) at 0.011 AUROC cost. BiomedCLIP debiasing is unreliable (p≥0.12), suggesting embedding expressivity is a prerequisite for demographic disentanglement.