Timezone: »

 
Spotlight
PMIC: Improving Multi-Agent Reinforcement Learning with Progressive Mutual Information Collaboration
Pengyi Li · Hongyao Tang · Tianpei Yang · Xiaotian Hao · Tong Sang · Yan Zheng · Jianye Hao · Matthew Taylor · Wenyuan Tao · Zhen Wang

Tue Jul 19 11:40 AM -- 11:45 AM (PDT) @ Room 309
in Reinforcement Learning: Deep/Batch/Offline »

Learning to collaborate is critical in Multi-Agent Reinforcement Learning (MARL). Previous works promote collaboration by maximizing the correlation of agents’ behaviors, which is typically characterized by Mutual Information (MI) in different forms. However, we reveal sub-optimal collaborative behaviors also emerge with strong correlations, and simply maximizing the MI can, surprisingly, hinder the learning towards better collaboration. To address this issue, we propose a novel MARL framework, called Progressive Mutual Information Collaboration (PMIC), for more effective MI-driven collaboration. PMIC uses a new collaboration criterion measured by the MI between global states and joint actions. Based on this criterion, the key idea of PMIC is maximizing the MI associated with superior collaborative behaviors and minimizing the MI associated with inferior ones. The two MI objectives play complementary roles by facilitating better collaborations while avoiding falling into sub-optimal ones. Experiments on a wide range of MARL benchmarks show the superior performance of PMIC compared with other algorithms.

Keywords: Reinforcement Learning · RL: Deep RL · RL: Multi-agent

Author Information

Pengyi Li (Tianjin university)
Hongyao Tang (Tianjin University)
Tianpei Yang (University of Alberta)
Xiaotian Hao (College of Intelligence and Computing, Tianjin University)
Tong Sang (Tianjin University)
Yan Zheng (Tianjin University, Nanyang Technical University)
Jianye Hao (Tianjin University)
Matthew Taylor (U. of Alberta)
Wenyuan Tao (Tianjin University)
Zhen Wang (Northwestern Polytechnical University)

Related Events (a corresponding poster, oral, or spotlight)

More from the Same Authors