Adversarial Training with Large Step Sizes: Implicit Bias and Evolution of Sharpness

Adversarial training (AT) can be modeled as a two-player zero-sum game. This game-theoretic characterization introduces additional challenges in analyzing its dynamics. Existing analyses of AT assume that the learner uses extremely small step sizes, which is unrealistic in practice. In this paper, we study AT dynamics under large step sizes, focusing on two aspects: the implicit bias of adversarial logistic regression and the sharpness evolution along AT trajectories. For the former, we show that adversarial logistic regression converges to a robust max-margin direction under arbitrary constant step sizes, generalizing the result of (Li et al., 2020), which requires an exponentially small step size. For the latter, we find that sharpness along AT trajectories exhibits a surprisingly regular pattern. Compared with the edge-of-stability phenomenon in standard training, this pattern contains an additional stage where sharpness oscillates while showing an overall decreasing trend.