Timezone: »

K-Beam Minimax: Efficient Optimization for Deep Adversarial Learning
Jihun Hamm · Yung-Kyun Noh

Thu Jul 12 09:15 AM -- 12:00 PM (PDT) @ Hall B #96
Minimax optimization plays a key role in adversarial training of machine learning algorithms, such as learning generative models, domain adaptation, privacy preservation, and robust learning. In this paper, we demonstrate the failure of alternating gradient descent in minimax optimization problems due to the discontinuity of solutions of the inner maximization. To address this, we propose a new $\epsilon$-subgradient descent algorithm that addresses this problem by simultaneously tracking $K$ candidate solutions. Practically, the algorithm can find solutions that previous saddle-point algorithms cannot find, with only a sublinear increase of complexity in $K$. We analyze the conditions under which the algorithm converges to the true solution in detail. A significant improvement in stability and convergence speed of the algorithm is observed in simple representative problems, GAN training, and domain-adaptation problems.

Author Information

Jihun Hamm (The Ohio State University)
Yung-Kyun Noh (Seoul National University)

Related Events (a corresponding poster, oral, or spotlight)