Posterior Mismatch Matters: Adversarial Training for Long-Tailed Robustness

Adversarial training breaks down in long-tailed settings, exhibiting severe robustness degradation on worst-performing (often tail) classes. We identify a key cause of this failure as a posterior mismatch: coarse-grained absolute labels collapse class posteriors into point estimates, leading to biased class-frequency estimation and an enlarged robust generalization gap, which ultimately amplifies worst-class vulnerability. To address this issue, we propose Posterior-driven Adversarial Training (PAT), which learns a posterior surrogate to provide fine-grained probabilistic supervision for adversarial training, and integrates weight perturbations to encourage a flatter loss landscape. Our theory shows that accurate posterior approximation simultaneously tightens class-frequency estimation error and robust generalization bounds, while a flat weight loss landscape stabilizes sensitivity to posterior approximation errors. Extensive experiments on long-tailed benchmarks confirm that PAT consistently improves robustness, with especially large gains on worst-class.

Modern AI systems are often vulnerable to adversarial attacks—small, carefully designed input perturbations that can cause incorrect predictions. This problem becomes even more severe in real-world datasets, where some classes have far fewer examples than others (known as long-tailed distributions). In such settings, existing adversarial training (defensive method) tends to perform poorly on rare classes, leading to significant adversarial robustness disparities. In this work, we identify a key reason for this failure: existing methods rely on hard labels that assign every sample to a single class with full confidence. We show that this oversimplified supervision creates a mismatch between the true uncertainty of the data and the labels used during training, harming adversarial robustness on rare classes. Our theoretical analysis shows that accurate posterior estimation and flat optimization landscapes are jointly important for solving this mismatch. Therefore, we propose Posterior-driven Adversarial Training (PAT). Instead of relying only on hard labels, PAT learns a surrogate model that estimates class uncertainty and provides soft supervision during adversarial training. PAT also incorporates weight perturbations to encourage flatter and more stable optimization landscapes. Extensive experiments on standard long-tailed benchmarks show that PAT consistently improves robustness, particularly for the worst-performing classes. These findings suggest that probabilistic supervision is an important direction for building more reliable and fair robust AI systems.