Timezone: »

Curious Replay for Model-based Adaptation
Isaac Kauvar · Chris Doyle · Linqi Zhou · Nick Haber

Wed Jul 26 02:00 PM -- 03:30 PM (PDT) @ Exhibit Hall 1 #618
Event URL: https://sites.google.com/view/curious-replay »

Agents must be able to adapt quickly as an environment changes. We find that existing model-based reinforcement learning agents are unable to do this well, in part because of how they use past experiences to train their world model. Here, we present Curious Replay---a form of prioritized experience replay tailored to model-based agents through use of a curiosity-based priority signal. Agents using Curious Replay exhibit improved performance in an exploration paradigm inspired by animal behavior and on the Crafter benchmark. DreamerV3 with Curious Replay surpasses state-of-the-art performance on Crafter, achieving a mean score of 19.4 that substantially improves on the previous high score of 14.5 by DreamerV3 with uniform replay, while also maintaining similar performance on the Deepmind Control Suite. Code for Curious Replay is available at github.com/AutonomousAgentsLab/curiousreplay.

Author Information

Isaac Kauvar (Stanford University)
Chris Doyle (Stanford University)
Linqi Zhou (Stanford University)
Nick Haber (Stanford University)

Assistant Professor at Stanford University. Interested in more autonomous, interactively learning AI (e.g. curiosity and self-supervised learning), cognitive models, and learning tools.

More from the Same Authors