MESH-HR: Multimodal Fusion of Histopathology and Structured Somatic Genomics for Continuous Breast Cancer Receptor Subtyping

Breast cancer receptor subtyping guides treatment and prognosis, as eligibility for targeted therapies is determined by estrogen receptor (ER), progesterone receptor (PR), and human epidermal growth factor receptor 2 (HER2) expression or amplification status. While these biomarkers are clinically assessed using immunohistochemistry (IHC), computational approaches have largely focused on directly predicting receptor status from histopathology images, despite complementary signal in somatic genomic profiles. We introduce MESH-HR (Multimodal Ensemble of Somatic Variants and H\&E Slides for Hormone Receptor subtyping), a model that integrates structured somatic genomic features and multi-resolution pathology images to predict ER, PR, and HER2 receptor status probabilities from routinely collected data, without requiring IHC. MESH-HR combines an attention-based multiple instance learning (ABMIL) vision encoder with an XGBoost model over structured genomics, capturing complementary signal across modalities. Trained on $>$1,300 breast cancers, MESH-HR achieves AUCs of 0.90 (ER), 0.84 (PR), and 0.96 (HER2) on held-out data, outperforming unimodal and prior imaging-only approaches, and generalizes zero-shot to The Cancer Genome Atlas Breast Cancer cohort (TCGA-BRCA). Continuous predictions improve survival stratification over binary outputs and discretized clinical IHC categories, recovering signal lost under discrete clinical labeling. We further apply MESH-HR to cancers of unknown primary (CUP), where receptor status is often unavailable, obtaining biologically consistent, survival-predictive subtype estimates that enable receptor-informed stratification.