Risk Perspective Exploration in Distributional Reinforcement Learning

Distributional reinforcement learning shows state-of-the-art performance in continuous and discrete control settings with the properties of variance and risk, which can be utilized as a means of exploration. However, the exploration method using risk property is hard to find, while various exploration methods use the variance of return distribution per action in Distributional RL. In this paper, we propose risk scheduling methods, a risk perspective exploration, that explore risk levels and optimistic actions. We show the performance improvement of DMIX algorithm by risk scheduling in a multi-agent setting with various experiments.