Timezone: »

 
Poster
Stability and Generalization of Stochastic Gradient Methods for Minimax Problems
Yunwen Lei · Zhenhuan Yang · Tianbao Yang · Yiming Ying

Wed Jul 21 09:00 PM -- 11:00 PM (PDT) @
in Poster Session 4 »

Many machine learning problems can be formulated as minimax problems such as Generative Adversarial Networks (GANs), AUC maximization and robust estimation, to mention but a few. A substantial amount of studies are devoted to studying the convergence behavior of their stochastic gradient-type algorithms. In contrast, there is relatively little work on understanding their generalization, i.e., how the learning models built from training examples would behave on test examples. In this paper, we provide a comprehensive generalization analysis of stochastic gradient methods for minimax problems under both convex-concave and nonconvex-nonconcave cases through the lens of algorithmic stability. We establish a quantitative connection between stability and several generalization measures both in expectation and with high probability. For the convex-concave setting, our stability analysis shows that stochastic gradient descent ascent attains optimal generalization bounds for both smooth and nonsmooth minimax problems. We also establish generalization bounds for both weakly-convex-weakly-concave and gradient-dominated problems. We report preliminary experimental results to verify our theory.

Keywords: Statistical Learning Theory

Author Information

Yunwen Lei (University of Birmingham)
Zhenhuan Yang (SUNY Albany)
Tianbao Yang (The University of Iowa)
Yiming Ying (SUNY Albany)

Related Events (a corresponding poster, oral, or spotlight)

More from the Same Authors