CAMP: Coherent Alignment of Multimodal Prototypes for Explainable Complementary Learning

Most multimodal learning assumes redundant views (such as image–caption pairs), yet many applications require combining complementary modalities that provide distinct evidence (such as an X-ray and medical history). We term this setting *Complementary Multimodal Classification* (CMC). In CMC, existing explainable-by-design methods often force an accuracy–interpretability trade-off because single shared similarity metrics fail under asymmetric, class-conditional evidence. To address this, we propose Coherent Alignment of Multimodal Prototypes (CAMP). CAMP enforces coherent multimodal reasoning by aligning class-wise evidence via optimal transport and imposing geometric constraints to counter modality dominance and representation collapse. We provide theoretical guarantees showing that these mechanisms eliminate such degeneracies without restricting expressivity. Empirically, across 16 public CMC datasets, CAMP matches or exceeds large ($>$100M parameter) AutoML baselines with fewer than 1M trainable parameters, and when fine-tuned end-to-end it achieves state-of-the-art performance. To the best of our knowledge, this work is the first modality-agnostic prototype-learning framework designed for complementary multimodal tasks.

Many machine-learning systems must combine different kinds of evidence, such as a medical image, a doctor’s notes, and patient metadata. These sources often complement one another rather than simply describing the same thing, but today’s models can be hard to audit and may lean too heavily on one source. We introduce CAMP, a model that predicts by comparing a new case with a small set of learned “representative cases.” CAMP decides how much weight to give each available source of information and can still run when some sources are missing. During training, CAMP keeps the representative cases for each class consistent across data sources while preventing them from becoming redundant, unused, or dominated by one source. This lets the model provide built-in evidence for its decisions: which sources mattered and which similar cases supported the prediction. Across 16 public multimodal benchmarks, a lightweight version of CAMP with fewer than 1 million trainable parameters matched or outperformed much larger automated systems on most tasks, and the fully fine-tuned version achieved the best results overall. CAMP shows that accurate multimodal prediction and inspectable, case-based explanations can be combined, which is especially important for healthcare, fraud detection, and online safety.