Class-Conditional Distribution Balancing for Group Robust Classification

Spurious correlations that lead models to correct predictions for the wrong reasons pose a critical challenge for robust real-world generalization. Existing research attributes this issue to group imbalance and addresses it by maximizing group-balanced or worst-group accuracy, which heavily relies on expensive bias annotations. A compromise approach involves predicting bias information using extensively pretrained foundation models, which requires large-scale data and is limited to physically interpretable biases. To address these challenges, we offer a novel perspective by reframing the spurious correlations as imbalances/mismatches in class-conditional distributions caused by general biases, whether interpretable or not, and propose a simple yet effective robust learning method that mitigates the need for bias annotations or predictions. With the goal of maximizing the conditional entropy (uncertainty) of the label given spurious factors, our method leverages a sample reweighting strategy to achieve class-conditional distribution balancing, which automatically highlights minority groups and classes, effectively mitigating spurious correlations and producing a debiased data distribution for classification. Extensive experiments and analysis demonstrate that our approach consistently delivers strong overall performance, and is competitive with methods that rely on bias supervision.

Machine learning models can sometimes make correct predictions for the wrong reasons, such as relying on accidental patterns in the data instead of the true underlying information. This can cause them to fail when applied to new, real-world situations. Many existing solutions try to fix this problem by using extra information about what types of bias are present in the data. However, collecting such information is often expensive, and in some cases not possible. In this paper, we introduce a simpler method that does not require any bias-related information. Our approach treats the problem as an imbalance in the data and automatically gives more importance to underrepresented examples during training. This helps the model learn more balanced patterns and reduces its reliance on misleading shortcuts in the data. Experiments show that our method improves robustness across multiple datasets and performs competitively with more complex approaches that rely on additional bias information.