Timezone: »
The real world is large and complex. It is filled with many objects besides those defined by a task and objects can move with their own interesting dynamics. How should an agent learn to represent state to support efficient learning and generalization in such an environment? In this work, we present a novel memory architecture, Perceptual Schemata, for learning and zero-shot generalization in environments that have many, potentially moving objects. Perceptual Schemata represents state using a combination of schema modules that each learn to attend to and maintain stateful representations of different subspaces of a spatio-temporal tensor describing the agent’s observations. We present empirical results that Perceptual Schemata enables a state representation that can maintain multiple objects observed in sequence with independent dynamics while an LSTM cannot. We additionally show that Perceptual Schemata can generalize more gracefully to larger environments with more distractor objects, while an LSTM quickly overfits to the training tasks.
Author Information
Wilka Carvalho (University of Michigan)
Murray Shanahan (DeepMind / Imperial College London)
More from the Same Authors
-
2021 : Learning to Represent State with Perceptual Schemata »
Wilka T Carvalho · Murray Shanahan -
2020 Poster: Learning to Combine Top-Down and Bottom-Up Signals in Recurrent Neural Networks with Attention over Modules »
Sarthak Mittal · Alex Lamb · Anirudh Goyal · Vikram Voleti · Murray Shanahan · Guillaume Lajoie · Michael Mozer · Yoshua Bengio -
2020 Poster: An Explicitly Relational Neural Network Architecture »
Murray Shanahan · Kyriacos Nikiforou · Antonia Creswell · Christos Kaplanis · David GT Barrett · Marta Garnelo -
2019 Poster: Policy Consolidation for Continual Reinforcement Learning »
Christos Kaplanis · Murray Shanahan · Claudia Clopath -
2019 Oral: Policy Consolidation for Continual Reinforcement Learning »
Christos Kaplanis · Murray Shanahan · Claudia Clopath -
2018 Poster: Continual Reinforcement Learning with Complex Synapses »
Christos Kaplanis · Murray Shanahan · Claudia Clopath -
2018 Poster: Conditional Neural Processes »
Marta Garnelo · Dan Rosenbaum · Chris Maddison · Tiago Ramalho · David Saxton · Murray Shanahan · Yee Teh · Danilo J. Rezende · S. M. Ali Eslami -
2018 Oral: Continual Reinforcement Learning with Complex Synapses »
Christos Kaplanis · Murray Shanahan · Claudia Clopath -
2018 Oral: Conditional Neural Processes »
Marta Garnelo · Dan Rosenbaum · Chris Maddison · Tiago Ramalho · David Saxton · Murray Shanahan · Yee Teh · Danilo J. Rezende · S. M. Ali Eslami