Timezone: »

 
Spotlight
The Power of Exploiter: Provable Multi-Agent RL in Large State Spaces
Chi Jin · Qinghua Liu · Tiancheng Yu

Wed Jul 20 07:50 AM -- 07:55 AM (PDT) @ Room 310
in T: Game Theory/RL/Planning »

Modern reinforcement learning (RL) commonly engages practical problems with large state spaces, where function approximation must be deployed to approximate either the value function or the policy. While recent progresses in RL theory address a rich set of RL problems with general function approximation, such successes are mostly restricted to the single-agent setting. It remains elusive how to extend these results to multi-agent RL, especially in the face of new game-theoretical challenges. This paper considers two-player zero-sum Markov Games (MGs). We propose a new algorithm that can provably find the Nash equilibrium policy using a polynomial number of samples, for any MG with low \emph{multi-agent Bellman-Eluder dimension}---a new complexity measure adapted from its single-agent version (Jin et al., 2021). A key component of our new algorithm is the exploiter, which facilitates the learning of the main player by deliberately exploiting her weakness. Our theoretical framework is generic, which applies to a wide range of models including but not limited to tabular MGs, MGs with linear or kernel function approximation, and MGs with rich observations.

Keywords: RL: Function Approximation · T: Reinforcement Learning and Planning

Author Information

Chi Jin (Princeton University)
Qinghua Liu (Princeton University)
Tiancheng Yu (MIT)

I am quant researcher at Two Sigma. I completed my PhD in MIT EECS in 2023.

Related Events (a corresponding poster, oral, or spotlight)

More from the Same Authors