Iterative Robust Satisficing: Minimizing Performance Degradation Under Distribution Shift

Modern neural networks often achieve high accuracy on their training distribution but degrade sharply under distribution shifts. We address this problem through Robust Satisficing (RS), an optimization objective that seeks parameters which attain a target level of in-distribution performance while minimizing fragility, defined as the rate at which performance deteriorates as the data distribution departs from training. We develop a gradient-based algorithm, Iterative Robust Satisficing (IRS), that directly optimizes this criterion. Across a range of synthetic and real-world distribution shifts, including long-tailed image classification, group shifts induced by spurious correlations, and natural shifts in tabular regression, IRS consistently improves performance on minority and worst-case groups without sacrificing overall accuracy. Notably, IRS achieves these robustness gains with a per-step computational cost similar to standard stochastic gradient descent and requires only a single forward and backward pass per update. Together, these results suggest that minimizing fragility provides a practical and effective alternative to existing robust training methods for learning models that remain reliable under distribution shift.