Timezone: »

 
Spotlight
Disentangling Sources of Risk for Distributional Multi-Agent Reinforcement Learning
Kyunghwan Son · Junsu Kim · Sungsoo Ahn · Roben Delos Reyes · Yung Yi · Jinwoo Shin

Tue Jul 19 01:30 PM -- 01:35 PM (PDT) @ Room 318 - 320
in RL: Multi-agent »

In cooperative multi-agent reinforcement learning, the outcomes of agent-wise policies are highly stochastic due to the two sources of risk: (a) random actions taken by teammates and (b) random transition and rewards. Although the two sources have very distinct characteristics, existing frameworks are insufficient to control the risk-sensitivity of agent-wise policies in a disentangled manner. To this end, we propose Disentangled RIsk-sensitive Multi-Agent reinforcement learning (DRIMA) to separately access the risk sources. For example, our framework allows an agent to be optimistic with respect to teammates (who can prosocially adapt) but more risk-neutral with respect to the environment (which does not adapt). Our experiments demonstrate that DRIMA significantly outperforms prior state-of-the-art methods across various scenarios in the StarCraft Multi-agent Challenge environment. Notably, DRIMA shows robust performance where prior methods learn only a highly suboptimal policy, regardless of reward shaping, exploration scheduling, and noisy (random or adversarial) agents.

Keywords: RL: Deep RL · RL: Risk Sensitive

Author Information

Kyunghwan Son (KAIST)
Junsu Kim (KAIST)
Sungsoo Ahn (POSTECH)
Roben Delos Reyes (Korea Advanced Institute of Science and Technology)
Yung Yi (KAIST)
Jinwoo Shin (KAIST)

Related Events (a corresponding poster, oral, or spotlight)

More from the Same Authors