Timezone: »

 
Oral
Meta-Learning Neural Bloom Filters
Jack Rae · Sergey Bartunov · Timothy Lillicrap

Thu Jun 13 11:25 AM -- 11:30 AM (PDT) @ Hall B
in Deep Sequence Models »

There has been a recent trend in training neural networks to replace data structures that have been crafted by hand, with an aim for faster execution, better accuracy, or greater compression. In this setting, a neural data structure is instantiated by training a network over many epochs of its inputs until convergence. In applications where inputs arrive at high throughput, or are ephemeral, training a network from scratch is not practical. This motivates the need for few-shot neural data structures. In this paper we explore the learning of approximate set membership over a set of data in one-shot via meta-learning. We propose a novel memory architecture, the Neural Bloom Filter, which is able to achieve significant compression gains over classical Bloom Filters and existing memory-augmented neural networks.

[ Slides] 

Author Information

Jack Rae (DeepMind)
Sergey Bartunov (DeepMind)
Timothy Lillicrap (Google DeepMind)

Related Events (a corresponding poster, oral, or spotlight)

More from the Same Authors

  • 2022 Poster: Retrieval-Augmented Reinforcement Learning »
    Anirudh Goyal · Abe Friesen Friesen · Andrea Banino · Theophane Weber · Nan Rosemary Ke · Adrià Puigdomenech Badia · Arthur Guez · Mehdi Mirza · Peter Humphreys · Ksenia Konyushkova · Michal Valko · Simon Osindero · Timothy Lillicrap · Nicolas Heess · Charles Blundell
  • 2022 Spotlight: Retrieval-Augmented Reinforcement Learning »
    Anirudh Goyal · Abe Friesen Friesen · Andrea Banino · Theophane Weber · Nan Rosemary Ke · Adrià Puigdomenech Badia · Arthur Guez · Mehdi Mirza · Peter Humphreys · Ksenia Konyushkova · Michal Valko · Simon Osindero · Timothy Lillicrap · Nicolas Heess · Charles Blundell
  • 2022 Poster: A data-driven approach for learning to control computers »
    Peter Humphreys · David Raposo · Tobias Pohlen · Gregory Thornton · Rachita Chhaparia · Alistair Muldal · Josh Abramson · Petko Georgiev · Adam Santoro · Timothy Lillicrap
  • 2022 Spotlight: A data-driven approach for learning to control computers »
    Peter Humphreys · David Raposo · Tobias Pohlen · Gregory Thornton · Rachita Chhaparia · Alistair Muldal · Josh Abramson · Petko Georgiev · Adam Santoro · Timothy Lillicrap
  • 2022 Poster: Improving Language Models by Retrieving from Trillions of Tokens »
    Sebastian Borgeaud · Arthur Mensch · Jordan Hoffmann · Trevor Cai · Eliza Rutherford · Katie Millican · George van den Driessche · Jean-Baptiste Lespiau · Bogdan Damoc · Aidan Clark · Diego de Las Casas · Aurelia Guy · Jacob Menick · Roman Ring · Tom Hennigan · Saffron Huang · Loren Maggiore · Chris Jones · Albin Cassirer · Andy Brock · Michela Paganini · Geoffrey Irving · Oriol Vinyals · Simon Osindero · Karen Simonyan · Jack Rae · Erich Elsen · Laurent Sifre
  • 2022 Poster: Unified Scaling Laws for Routed Language Models »
    Aidan Clark · Diego de Las Casas · Aurelia Guy · Arthur Mensch · Michela Paganini · Jordan Hoffmann · Bogdan Damoc · Blake Hechtman · Trevor Cai · Sebastian Borgeaud · George van den Driessche · Eliza Rutherford · Tom Hennigan · Matthew Johnson · Albin Cassirer · Chris Jones · Elena Buchatskaya · David Budden · Laurent Sifre · Simon Osindero · Oriol Vinyals · Marc'Aurelio Ranzato · Jack Rae · Erich Elsen · Koray Kavukcuoglu · Karen Simonyan
  • 2022 Spotlight: Improving Language Models by Retrieving from Trillions of Tokens »
    Sebastian Borgeaud · Arthur Mensch · Jordan Hoffmann · Trevor Cai · Eliza Rutherford · Katie Millican · George van den Driessche · Jean-Baptiste Lespiau · Bogdan Damoc · Aidan Clark · Diego de Las Casas · Aurelia Guy · Jacob Menick · Roman Ring · Tom Hennigan · Saffron Huang · Loren Maggiore · Chris Jones · Albin Cassirer · Andy Brock · Michela Paganini · Geoffrey Irving · Oriol Vinyals · Simon Osindero · Karen Simonyan · Jack Rae · Erich Elsen · Laurent Sifre
  • 2022 Oral: Unified Scaling Laws for Routed Language Models »
    Aidan Clark · Diego de Las Casas · Aurelia Guy · Arthur Mensch · Michela Paganini · Jordan Hoffmann · Bogdan Damoc · Blake Hechtman · Trevor Cai · Sebastian Borgeaud · George van den Driessche · Eliza Rutherford · Tom Hennigan · Matthew Johnson · Albin Cassirer · Chris Jones · Elena Buchatskaya · David Budden · Laurent Sifre · Simon Osindero · Oriol Vinyals · Marc'Aurelio Ranzato · Jack Rae · Erich Elsen · Koray Kavukcuoglu · Karen Simonyan
  • 2020 Poster: Stabilizing Transformers for Reinforcement Learning »
    Emilio Parisotto · Francis Song · Jack Rae · Razvan Pascanu · Caglar Gulcehre · Siddhant Jayakumar · Max Jaderberg · Raphael Lopez Kaufman · Aidan Clark · Seb Noury · Matthew Botvinick · Nicolas Heess · Raia Hadsell
  • 2019 Poster: Learning Latent Dynamics for Planning from Pixels »
    Danijar Hafner · Timothy Lillicrap · Ian Fischer · Ruben Villegas · David Ha · Honglak Lee · James Davidson
  • 2019 Oral: Learning Latent Dynamics for Planning from Pixels »
    Danijar Hafner · Timothy Lillicrap · Ian Fischer · Ruben Villegas · David Ha · Honglak Lee · James Davidson
  • 2019 Poster: Deep Compressed Sensing »
    Yan Wu · Mihaela Rosca · Timothy Lillicrap
  • 2019 Oral: Deep Compressed Sensing »
    Yan Wu · Mihaela Rosca · Timothy Lillicrap
  • 2019 Poster: Composing Entropic Policies using Divergence Correction »
    Jonathan Hunt · Andre Barreto · Timothy Lillicrap · Nicolas Heess
  • 2019 Poster: An Investigation of Model-Free Planning »
    Arthur Guez · Mehdi Mirza · Karol Gregor · Rishabh Kabra · Sebastien Racaniere · Theophane Weber · David Raposo · Adam Santoro · Laurent Orseau · Tom Eccles · Greg Wayne · David Silver · Timothy Lillicrap
  • 2019 Oral: An Investigation of Model-Free Planning »
    Arthur Guez · Mehdi Mirza · Karol Gregor · Rishabh Kabra · Sebastien Racaniere · Theophane Weber · David Raposo · Adam Santoro · Laurent Orseau · Tom Eccles · Greg Wayne · David Silver · Timothy Lillicrap
  • 2019 Oral: Composing Entropic Policies using Divergence Correction »
    Jonathan Hunt · Andre Barreto · Timothy Lillicrap · Nicolas Heess
  • 2018 Poster: Measuring abstract reasoning in neural networks »
    Adam Santoro · Feilx Hill · David GT Barrett · Ari S Morcos · Timothy Lillicrap
  • 2018 Oral: Measuring abstract reasoning in neural networks »
    Adam Santoro · Feilx Hill · David GT Barrett · Ari S Morcos · Timothy Lillicrap
  • 2018 Poster: Fast Parametric Learning with Activation Memorization »
    Jack Rae · Chris Dyer · Peter Dayan · Timothy Lillicrap
  • 2018 Oral: Fast Parametric Learning with Activation Memorization »
    Jack Rae · Chris Dyer · Peter Dayan · Timothy Lillicrap
  • 2017 Poster: Learning to Learn without Gradient Descent by Gradient Descent »
    Yutian Chen · Matthew Hoffman · Sergio Gómez Colmenarejo · Misha Denil · Timothy Lillicrap · Matthew Botvinick · Nando de Freitas
  • 2017 Talk: Learning to Learn without Gradient Descent by Gradient Descent »
    Yutian Chen · Matthew Hoffman · Sergio Gómez Colmenarejo · Misha Denil · Timothy Lillicrap · Matthew Botvinick · Nando de Freitas