Group Contrastive Learning for Weakly Paired Multimodal Data

We present GROOVE, a semi-supervised representation learning approach for weakly paired multi-modal data, where samples across modalities share perturbation labels but lack instance-level correspondence. Our central contribution is GroupCLIP, a group-level contrastive loss that fills the gap between CLIP (cross-modal, instance-paired) and SupCon (uni-modal, label-supervised) for the weakly-paired regime. We integrate GroupCLIP with on-the-fly backtranslating autoencoders to learn cross-modally entangled, group-coherent representations. We additionally propose a combinatorial benchmarking framework that pairs representation learners with multiple optimal-transport aligners, revealing that no single aligner uniformly dominates. Across simulations and two single-cell genetic perturbation datasets, GROOVE matches or outperforms existing approaches on cross-modal matching and imputation, with ablations showing GroupCLIP as the key driver of performance gains.