Injecting Distributional Awareness into MLLMs via Reinforcement Learning for Deep Imbalanced Regression

Multimodal large language models (MLLMs) struggle with numerical regression under longtailed target distributions. Token-level supervised fine-tuning (SFT) and point-wise regression rewards bias learning toward high-density regions, leading to regression-to-the-mean behavior and poor tail performance. We identify the lack of cross-sample relational supervision as a key limitation of existing MLLM training paradigms. To address it, we propose a distribution-aware reinforcement learning framework based on Group Relative Policy Optimization, which introduces batch-level comparison-based supervision via the Concordance Correlation Coefficient-based reward to align predicted and ground-truth distributions in terms of correlation, scale, and mean. The framework is plug-and-play, requiring no architectural modification. Experiments on a unified suite of long-tailed regression benchmarks show consistent improvements over SFT and existing MLLM regression methods, with particularly strong gains in medium- and few-shot regimes.

Multimodal large language models can describe images well, but they are much less reliable when asked to predict numbers, especially when rare values appear much less often than common ones in the training data. In these cases, the models tend to play it safe and predict values near the average, which hurts performance on unusual but important cases. We address this problem with a new training method that teaches the model to pay attention not only to each example on its own, but also to how its predictions compare across a group of examples. Instead of rewarding only point-by-point accuracy, our method rewards the model for matching the overall shape of the true distribution, including trends, spread, and average values. This can be added to existing models without changing their architecture. Across several long-tailed regression benchmarks, our method improves prediction accuracy over standard training and existing regression methods, with especially strong gains on medium- and low-frequency cases. This could help make multimodal AI systems more reliable for real-world tasks that require numerical predictions, such as age estimation, rating prediction, and medical assessment.