Timezone: »
We propose a method for tackling catastrophic forgetting in deep reinforcement learning that is \textit{agnostic} to the timescale of changes in the distribution of experiences, does not require knowledge of task boundaries and can adapt in \textit{continuously} changing environments. In our \textit{policy consolidation} model, the policy network interacts with a cascade of hidden networks that simultaneously remember the agent's policy at a range of timescales and regularise the current policy by its own history, thereby improving its ability to learn without forgetting. We find that the model improves continual learning relative to baselines on a number of continuous control tasks in single-task, alternating two-task, and multi-agent competitive self-play settings.
Author Information
Christos Kaplanis (Imperial College London)
PhD student investigating the topic of continual learning in artificial neural networks.
Murray Shanahan (DeepMind / Imperial College London)
Claudia Clopath (Imperial College London)
Related Events (a corresponding poster, oral, or spotlight)
-
2019 Oral: Policy Consolidation for Continual Reinforcement Learning »
Tue. Jun 11th 07:00 -- 07:05 PM Room Hall B
More from the Same Authors
-
2021 : Learning to Represent State with Perceptual Schemata »
Wilka T Carvalho · Murray Shanahan -
2021 : Learning to Represent State with Perceptual Schemata »
Wilka Carvalho · Murray Shanahan -
2022 Poster: Maslow's Hammer in Catastrophic Forgetting: Node Re-Use vs. Node Activation »
Sebastian Lee · Stefano Sarao Mannelli · Claudia Clopath · Sebastian Goldt · Andrew Saxe -
2022 Spotlight: Maslow's Hammer in Catastrophic Forgetting: Node Re-Use vs. Node Activation »
Sebastian Lee · Stefano Sarao Mannelli · Claudia Clopath · Sebastian Goldt · Andrew Saxe -
2021 Poster: Spectral Normalisation for Deep Reinforcement Learning: An Optimisation Perspective »
Florin Gogianu · Tudor Berariu · Mihaela Rosca · Claudia Clopath · Lucian Busoniu · Razvan Pascanu -
2021 Spotlight: Spectral Normalisation for Deep Reinforcement Learning: An Optimisation Perspective »
Florin Gogianu · Tudor Berariu · Mihaela Rosca · Claudia Clopath · Lucian Busoniu · Razvan Pascanu -
2020 : Invited Talk: Claudia Clopath "Continual learning though consolidation – a neuroscience angle" »
Claudia Clopath -
2020 Poster: Learning to Combine Top-Down and Bottom-Up Signals in Recurrent Neural Networks with Attention over Modules »
Sarthak Mittal · Alex Lamb · Anirudh Goyal · Vikram Voleti · Murray Shanahan · Guillaume Lajoie · Michael Mozer · Yoshua Bengio -
2020 Poster: An Explicitly Relational Neural Network Architecture »
Murray Shanahan · Kyriacos Nikiforou · Antonia Creswell · Christos Kaplanis · David GT Barrett · Marta Garnelo -
2018 Poster: Continual Reinforcement Learning with Complex Synapses »
Christos Kaplanis · Murray Shanahan · Claudia Clopath -
2018 Poster: Conditional Neural Processes »
Marta Garnelo · Dan Rosenbaum · Chris Maddison · Tiago Ramalho · David Saxton · Murray Shanahan · Yee Teh · Danilo J. Rezende · S. M. Ali Eslami -
2018 Oral: Continual Reinforcement Learning with Complex Synapses »
Christos Kaplanis · Murray Shanahan · Claudia Clopath -
2018 Oral: Conditional Neural Processes »
Marta Garnelo · Dan Rosenbaum · Chris Maddison · Tiago Ramalho · David Saxton · Murray Shanahan · Yee Teh · Danilo J. Rezende · S. M. Ali Eslami