Timezone: »

 
Poster
Open-ended learning in symmetric zero-sum games
David Balduzzi · Marta Garnelo · Yoram Bachrach · Wojciech Czarnecki · Julien Perolat · Max Jaderberg · Thore Graepel

Tue Jun 11 06:30 PM -- 09:00 PM (PDT) @ Pacific Ballroom #158
in Posters Tue »

Zero-sum games such as chess and poker are, abstractly, functions that evaluate pairs of agents, for example labeling them winner' andloser'. If the game is approximately transitive, then self-play generates sequences of agents of increasing strength. However, nontransitive games, such as rock-paper-scissors, can exhibit strategic cycles, and there is no longer a clear objective -- we want agents to increase in strength, but against whom is unclear. In this paper, we introduce a geometric framework for formulating agent objectives in zero-sum games, in order to construct adaptive sequences of objectives that yield open-ended learning. The framework allows us to reason about population performance in nontransitive games, and enables the development of a new algorithm (rectified Nash response, PSROrN) that uses game-theoretic niching to construct diverse populations of effective agents, producing a stronger set of agents than existing algorithms. We apply PSROrN to two highly nontransitive resource allocation games and find that PSRO_rN consistently outperforms the existing alternatives.

Keywords: Game Theory and Mechanism Design · Multiagent Learning

Author Information

David Balduzzi (DeepMind)
Marta Garnelo (DeepMind)
Yoram Bachrach
Wojciech Czarnecki (DeepMind)
Julien Perolat (DeepMind)
Max Jaderberg (DeepMind)
Thore Graepel (DeepMind)

Related Events (a corresponding poster, oral, or spotlight)

More from the Same Authors