OmniSapiens: A Foundation Model for Social Behavior Processing via Heterogeneity-Aware Relative Policy Optimization

Socially intelligent AI systems must reason across diverse human behavioral tasks and generalize to new social contexts. However, behavioral data is inherently heterogeneous, comprising diverse modalities and prediction targets that produce uneven training signals across samples, creating imbalanced learning dynamics that challenge existing AI models. To address this, we develop OmniSapiens-7B 2.0, a foundation model for social behavior processing that explicitly addresses learning from heterogeneous behavioral data. This is enabled through Heterogeneity-Aware Relative Policy Optimization, a new RL method that rebalances learning signals across samples by approximating each sample’s contribution to the policy update and using these estimates to drive geometrically centered, inertially smoothed advantage modulation for stable training. OmniSapiens-7B 2.0 achieves the best and most consistent performance across 10 behavioral tasks, while also attaining the best performance on all five held-out benchmarks, with gains of up to +12.02% and +9.37% respectively. Furthermore, it demonstrates more consistent and interpretable reasoning traces, supporting reliable real-world behavioral applications. Our model is available at https://github.com/MIT-MI/humanbehavioratlas.

People are socially complex, we read tone, gestures, context, and intent all at once. Building AI that understands human behavior this way is hard because the data is varied and uneven: it spans video, audio, and text, and the tasks range from recognizing emotions to inferring someone's intentions. When AI models train on this data mix, some data samples dominate the learning process while others get drowned out, affecting the model's performance. We built OmniSapiens-7B 2.0, an AI model designed specifically to learn from this kind of varied and diverse social data. The key idea is a new training method that controls how much each data sample is influences the model's learning, then adjusts the influence of each sample so that no single type of data dominates. Think of it like an orchestra conductor balancing the volume of each section, so the brass doesn't drown out the strings. The result is an AI model that better understands human behavior across the board. It performs best on 10 different social-behavior tasks, and also handles 5 completely new tasks it wasn't trained on, with improvements of up to 12% over previous models. It also shows its reasoning processes more consistently and clearly, which provides us an improved interpretation of its thinking process. The model and code are open-source, available at the linked GitHub: https://github.com/MIT-MI/humanbehavioratlas.