Enhancing Cross-subject Emotion Recognition via Heterogeneous Distribution Augmentation and Collaborative Learning

Cross-subject emotion recognition aims to improve a model's generalization to previously unseen subjects. Existing methods are mainly built upon domain generalization or data augmentation, but suffer from two major limitations: 1) heavy dependence on modality-specific feature designs—almost exclusively tailored to EEG signals—resulting in limited generalizability; and 2) the widespread assumption of independently and identically distributed data, which restricts the diversity of generated samples. To address these challenges, we systematically analyze the heterogeneous distribution characteristics of emotion data and propose MixEmo, a framework that integrates heterogeneous distribution augmentation and collaborative learning. Specifically, a well-trained backbone is used to extract representations and partition them into multiple single-distribution subsets as distribution prototypes. These prototypes are randomly combined to synthesize unseen distributions, thereby enhancing distributional diversity. Finally, heterogeneous distribution collaborative learning jointly optimizes the model across subsets. Extensive experiments demonstrate that MixEmo substantially improves generalization performance in cross-subject emotion recognition.

Emotion recognition systems often struggle to work well for new users whose data were not seen during training. Existing approaches usually rely on methods designed for specific types of signals, especially brain activity signals, making them difficult to apply broadly. In addition, they often assume that all training data follow similar patterns, which limits the diversity of the data used to improve the model. To address these issues, we propose MixEmo, a framework that helps models learn from a wider variety of emotional patterns. MixEmo first groups emotional data into several representative patterns and then mixes these patterns to create new and more diverse training examples. It also trains the model to learn collaboratively from different groups of data so that it can better adapt to unseen users. Experiments on multiple datasets show that MixEmo significantly improves the ability of emotion recognition systems to generalize to new subjects.