Timezone: »

Policy Diagnosis via Measuring Role Diversity in Cooperative Multi-agent RL
Siyi Hu · Chuanlong Xie · Xiaodan Liang · Xiaojun Chang

Thu Jul 21 01:50 PM -- 01:55 PM (PDT) @ Room 307

Cooperative multi-agent reinforcement learning (MARL) is making rapid progress for solving tasks in a grid world and real-world scenarios, in which agents are given different attributes and goals, resulting in different behavior through the whole multi-agent task. In this study, we quantify the agent's behavior difference and build its relationship with the policy performance via {\bf Role Diversity}, a metric to measure the characteristics of MARL tasks. We define role diversity from three perspectives: action-based, trajectory-based, and contribution-based to fully measure a multi-agent task. Through theoretical analysis, we find that the error bound in MARL can be decomposed into three parts that have a strong relation to the role diversity. The decomposed factors can significantly impact policy optimization in three popular directions including parameter sharing, communication mechanism, and credit assignment. The main experimental platforms are based on {\bf Multiagent Particle Environment (MPE) }and {\bf The StarCraft Multi-Agent Challenge (SMAC)}. Extensive experiments clearly show that role diversity can serve as a robust measurement for the characteristics of a multi-agent cooperation task and help diagnose whether the policy fits the current multi-agent system for better policy performance.

Author Information

Siyi Hu (Monash University)
Chuanlong Xie (Beijing Normal University)
Xiaodan Liang (Sun Yat-sen University)
Xiaojun Chang (University of Technology Sydney)

Related Events (a corresponding poster, oral, or spotlight)

More from the Same Authors