Timezone: »

Coordinated Exploration in Concurrent Reinforcement Learning
Maria Dimakopoulou · Benjamin Van Roy

Wed Jul 11 04:30 AM -- 04:50 AM (PDT) @ A1

We consider a team of reinforcement learning agents that concurrently learn to operate in a common environment. We identify three properties - adaptivity, commitment, and diversity - which are necessary for efficient coordinated exploration and demonstrate that straightforward extensions to single-agent optimistic and posterior sampling approaches fail to satisfy them. As an alternative, we propose seed sampling, which extends posterior sampling in a manner that meets these requirements. Simulation results investigate how per-agent regret decreases as the number of agents grows, establishing substantial advantages of seed sampling over alternative exploration schemes.

Author Information

Maria Dimakopoulou (Stanford)
Benjamin Van Roy (Stanford University)

Related Events (a corresponding poster, oral, or spotlight)

More from the Same Authors