Timezone: »
Learning data representations that are useful for various downstream tasks is a cornerstone of artificial intelligence. While existing methods are typically evaluated on downstream tasks such as classification or generative image quality, we propose to assess representations through their usefulness in downstream control tasks, such as reaching or pushing objects. By training over 10,000 reinforcement learning policies, we extensively evaluate to what extent different representation properties affect out-of-distribution (OOD) generalization. Finally, we demonstrate zero-shot transfer of these policies from simulation to the real world, without any domain randomization or fine-tuning. This paper aims to establish the first systematic characterization of the usefulness of learned representations for real-world OOD downstream tasks.
Author Information
Frederik Träuble (MPI for Intelligent Systems)
Andrea Dittadi (Technical University of Denmark)
Manuel Wuthrich (Max Planck Institute for Intelligent Systems)
Felix Widmaier (, Max Planck Institute for Intelligent Systems)
Peter V Gehler (Amazon)
Ole Winther (University of Copenhagen)
Francesco Locatello (Amazon)
Olivier Bachem (Google Brain)
Bernhard Schölkopf (MPI for Intelligent Systems Tübingen, Germany)
Bernhard Scholkopf received degrees in mathematics (London) and physics (Tubingen), and a doctorate in computer science from the Technical University Berlin. He has researched at AT&T Bell Labs, at GMD FIRST, Berlin, at the Australian National University, Canberra, and at Microsoft Research Cambridge (UK). In 2001, he was appointed scientific member of the Max Planck Society and director at the MPI for Biological Cybernetics; in 2010 he founded the Max Planck Institute for Intelligent Systems. For further information, see www.kyb.tuebingen.mpg.de/~bs.
Stefan Bauer (Max Planck Institute for Intelligent Systems)
More from the Same Authors
-
2021 : On the Fairness of Causal Algorithmic Recourse »
Julius von Kügelgen · Amir Karimi · Umang Bhatt · Isabel Valera · Adrian Weller · Bernhard Schölkopf · Amir-Hossein Karimi -
2021 : Algorithmic Recourse in Partially and Fully Confounded Settings Through Bounding Counterfactual Effects »
Julius von Kügelgen · Nikita Agarwal · Jakob Zeitler · Afsaneh Mastouri · Bernhard Schölkopf -
2021 : A functional mirror ascent view of policy gradient methods with function approximation »
Sharan Vaswani · Olivier Bachem · Simone Totaro · Matthieu Geist · Marlos C. Machado · Pablo Samuel Castro · Nicolas Le Roux -
2021 : Representation Learning for Out-of-distribution Generalization in Downstream Tasks »
Frederik Träuble · Andrea Dittadi · Manuel Wüthrich · Felix Widmaier · Peter Gehler · Ole Winther · Francesco Locatello · Olivier Bachem · Bernhard Schölkopf · Stefan Bauer -
2021 : Offline Reinforcement Learning as Anti-Exploration »
Shideh Rezaeifar · Robert Dadashi · Nino Vieillard · Léonard Hussenot · Olivier Bachem · Olivier Pietquin · Matthieu Geist -
2021 : Lie interventions in complex systems with cycles »
Michel Besserve · Bernhard Schölkopf -
2022 : Latent Variable Models for Bayesian Causal Discovery »
Jithendaraa Subramanian · Jithendaraa Subramanian · Yashas Annadani · Ivaxi Sheth · Stefan Bauer · Derek Nowrouzezahrai · Samira Ebrahimi Kahou -
2022 : On the Generalization and Adaption Performance of Causal Models »
Nino Scherrer · Anirudh Goyal · Stefan Bauer · Yoshua Bengio · Rosemary Nan Ke -
2022 : ImageNet-D: A new challenging robustness dataset inspired by domain adaptation »
Evgenia Rusak · Steffen Schneider · Peter V Gehler · Oliver Bringmann · Wieland Brendel · Matthias Bethge -
2022 : Maximum Mean Discrepancy Distributionally Robust Nonlinear Chance-Constrained Optimization with Finite-Sample Guarantee »
Yassine Nemmour · Heiner Kremer · Bernhard Schölkopf · Jia-Jie Zhu -
2022 : Deriving modular inductive biases from the principle of independent mechanisms »
Francesco Locatello -
2022 : Inductive Biases for Object-Centric Representations in the Presence of Complex Textures »
Samuele Papa · Ole Winther · Andrea Dittadi -
2022 : ImageNet-D: A new challenging robustness dataset inspired by domain adaptation »
Evgenia Rusak · Steffen Schneider · Peter V Gehler · Oliver Bringmann · Wieland Brendel · Matthias Bethge -
2022 : Invited talks I, Q/A »
Bernhard Schölkopf · David Lopez-Paz -
2022 : Invited Talks 1, Bernhard Schölkopf and David Lopez-Paz »
Bernhard Schölkopf · David Lopez-Paz -
2022 Poster: Action-Sufficient State Representation Learning for Control with Structural Constraints »
Biwei Huang · Chaochao Lu · Liu Leqi · Jose Miguel Hernandez-Lobato · Clark Glymour · Bernhard Schölkopf · Kun Zhang -
2022 Poster: Generalization and Robustness Implications in Object-Centric Learning »
Andrea Dittadi · Samuele Papa · Michele De Vita · Bernhard Schölkopf · Ole Winther · Francesco Locatello -
2022 Spotlight: Action-Sufficient State Representation Learning for Control with Structural Constraints »
Biwei Huang · Chaochao Lu · Liu Leqi · Jose Miguel Hernandez-Lobato · Clark Glymour · Bernhard Schölkopf · Kun Zhang -
2022 Spotlight: Generalization and Robustness Implications in Object-Centric Learning »
Andrea Dittadi · Samuele Papa · Michele De Vita · Bernhard Schölkopf · Ole Winther · Francesco Locatello -
2022 Poster: Adaptive Gaussian Process Change Point Detection »
Edoardo Caldarelli · Philippe Wenk · Stefan Bauer · Andreas Krause -
2022 Poster: Causal Inference Through the Structural Causal Marginal Problem »
Luigi Gresele · Julius von Kügelgen · Jonas Kübler · Elke Kirschbaum · Bernhard Schölkopf · Dominik Janzing -
2022 Poster: Functional Generalized Empirical Likelihood Estimation for Conditional Moment Restrictions »
Heiner Kremer · Jia-Jie Zhu · Krikamol Muandet · Bernhard Schölkopf -
2022 Poster: On the Adversarial Robustness of Causal Algorithmic Recourse »
Ricardo Dominguez-Olmedo · Amir Karimi · Bernhard Schölkopf -
2022 Spotlight: Functional Generalized Empirical Likelihood Estimation for Conditional Moment Restrictions »
Heiner Kremer · Jia-Jie Zhu · Krikamol Muandet · Bernhard Schölkopf -
2022 Spotlight: Causal Inference Through the Structural Causal Marginal Problem »
Luigi Gresele · Julius von Kügelgen · Jonas Kübler · Elke Kirschbaum · Bernhard Schölkopf · Dominik Janzing -
2022 Spotlight: On the Adversarial Robustness of Causal Algorithmic Recourse »
Ricardo Dominguez-Olmedo · Amir Karimi · Bernhard Schölkopf -
2022 Spotlight: Adaptive Gaussian Process Change Point Detection »
Edoardo Caldarelli · Philippe Wenk · Stefan Bauer · Andreas Krause -
2022 : Q&A »
Nan Rosemary Ke · Stefan Bauer -
2022 : Deep Learning for Causality »
Stefan Bauer -
2022 Tutorial: Causality and Deep Learning: Synergies, Challenges and the Future »
Nan Rosemary Ke · Stefan Bauer -
2021 Poster: Function Contrastive Learning of Transferable Meta-Representations »
Muhammad Waleed Gondal · Shruti Joshi · Nasim Rahaman · Stefan Bauer · Manuel Wuthrich · Bernhard Schölkopf -
2021 Spotlight: Function Contrastive Learning of Transferable Meta-Representations »
Muhammad Waleed Gondal · Shruti Joshi · Nasim Rahaman · Stefan Bauer · Manuel Wuthrich · Bernhard Schölkopf -
2021 Poster: On Disentangled Representations Learned from Correlated Data »
Frederik Träuble · Elliot Creager · Niki Kilbertus · Francesco Locatello · Andrea Dittadi · Anirudh Goyal · Bernhard Schölkopf · Stefan Bauer -
2021 Poster: Bayesian Quadrature on Riemannian Data Manifolds »
Christian Fröhlich · Alexandra Gessner · Philipp Hennig · Bernhard Schölkopf · Georgios Arvanitidis -
2021 Spotlight: Bayesian Quadrature on Riemannian Data Manifolds »
Christian Fröhlich · Alexandra Gessner · Philipp Hennig · Bernhard Schölkopf · Georgios Arvanitidis -
2021 Oral: On Disentangled Representations Learned from Correlated Data »
Frederik Träuble · Elliot Creager · Niki Kilbertus · Francesco Locatello · Andrea Dittadi · Anirudh Goyal · Bernhard Schölkopf · Stefan Bauer -
2021 Poster: Necessary and sufficient conditions for causal feature selection in time series with latent common causes »
Atalanti Mastakouri · Bernhard Schölkopf · Dominik Janzing -
2021 Poster: Conditional Distributional Treatment Effect with Kernel Conditional Mean Embeddings and U-Statistic Regression »
Junhyung Park · Uri Shalit · Bernhard Schölkopf · Krikamol Muandet -
2021 Spotlight: Necessary and sufficient conditions for causal feature selection in time series with latent common causes »
Atalanti Mastakouri · Bernhard Schölkopf · Dominik Janzing -
2021 Spotlight: Conditional Distributional Treatment Effect with Kernel Conditional Mean Embeddings and U-Statistic Regression »
Junhyung Park · Uri Shalit · Bernhard Schölkopf · Krikamol Muandet -
2021 Poster: Causal Curiosity: RL Agents Discovering Self-supervised Experiments for Causal Representation Learning »
Sumedh Sontakke · Arash Mehrjou · Laurent Itti · Bernhard Schölkopf -
2021 Poster: Hyperparameter Selection for Imitation Learning »
Léonard Hussenot · Marcin Andrychowicz · Damien Vincent · Robert Dadashi · Anton Raichuk · Sabela Ramos · Nikola Momchev · Sertan Girgin · Raphael Marinier · Lukasz Stafiniak · Emmanuel Orsini · Olivier Bachem · Matthieu Geist · Olivier Pietquin -
2021 Oral: Hyperparameter Selection for Imitation Learning »
Léonard Hussenot · Marcin Andrychowicz · Damien Vincent · Robert Dadashi · Anton Raichuk · Sabela Ramos · Nikola Momchev · Sertan Girgin · Raphael Marinier · Lukasz Stafiniak · Emmanuel Orsini · Olivier Bachem · Matthieu Geist · Olivier Pietquin -
2021 Spotlight: Causal Curiosity: RL Agents Discovering Self-supervised Experiments for Causal Representation Learning »
Sumedh Sontakke · Arash Mehrjou · Laurent Itti · Bernhard Schölkopf -
2020 Workshop: Inductive Biases, Invariances and Generalization in Reinforcement Learning »
Anirudh Goyal · Rosemary Nan Ke · Jane Wang · Stefan Bauer · Theophane Weber · Fabio Viola · Bernhard Schölkopf · Stefan Bauer -
2020 Poster: Weakly-Supervised Disentanglement Without Compromises »
Francesco Locatello · Ben Poole · Gunnar Ratsch · Bernhard Schölkopf · Olivier Bachem · Michael Tschannen -
2020 Poster: Automatic Shortcut Removal for Self-Supervised Representation Learning »
Matthias Minderer · Olivier Bachem · Neil Houlsby · Michael Tschannen -
2019 Poster: Robustly Disentangled Causal Mechanisms: Validating Deep Representations for Interventional Robustness »
Raphael Suter · Djordje Miladinovic · Bernhard Schölkopf · Stefan Bauer -
2019 Oral: Robustly Disentangled Causal Mechanisms: Validating Deep Representations for Interventional Robustness »
Raphael Suter · Djordje Miladinovic · Bernhard Schölkopf · Stefan Bauer -
2019 Poster: Kernel Mean Matching for Content Addressability of GANs »
Wittawat Jitkrittum · Wittawat Jitkrittum · Patsorn Sangkloy · Muhammad Waleed Gondal · Amit Raj · James Hays · Bernhard Schölkopf -
2019 Oral: Kernel Mean Matching for Content Addressability of GANs »
Wittawat Jitkrittum · Wittawat Jitkrittum · Patsorn Sangkloy · Patsorn Sangkloy · Muhammad Waleed Gondal · Muhammad Waleed Gondal · Amit Raj · Amit Raj · James Hays · James Hays · Bernhard Schölkopf · Bernhard Schölkopf -
2019 Poster: First-Order Adversarial Vulnerability of Neural Networks and Input Dimension »
Carl-Johann Simon-Gabriel · Yann Ollivier · Leon Bottou · Bernhard Schölkopf · David Lopez-Paz -
2019 Poster: Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representations »
Francesco Locatello · Stefan Bauer · Mario Lucic · Gunnar Ratsch · Sylvain Gelly · Bernhard Schölkopf · Olivier Bachem -
2019 Poster: High-Fidelity Image Generation With Fewer Labels »
Mario Lucic · Michael Tschannen · Marvin Ritter · Xiaohua Zhai · Olivier Bachem · Sylvain Gelly -
2019 Oral: High-Fidelity Image Generation With Fewer Labels »
Mario Lucic · Michael Tschannen · Marvin Ritter · Xiaohua Zhai · Olivier Bachem · Sylvain Gelly -
2019 Oral: First-Order Adversarial Vulnerability of Neural Networks and Input Dimension »
Carl-Johann Simon-Gabriel · Yann Ollivier · Leon Bottou · Bernhard Schölkopf · David Lopez-Paz -
2019 Oral: Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representations »
Francesco Locatello · Stefan Bauer · Mario Lucic · Gunnar Ratsch · Sylvain Gelly · Bernhard Schölkopf · Olivier Bachem -
2018 Poster: Detecting non-causal artifacts in multivariate linear regression models »
Dominik Janzing · Bernhard Schölkopf -
2018 Poster: On Matching Pursuit and Coordinate Descent »
Francesco Locatello · Anant Raj · Sai Praneeth Reddy Karimireddy · Gunnar Ratsch · Bernhard Schölkopf · Sebastian Stich · Martin Jaggi -
2018 Oral: Detecting non-causal artifacts in multivariate linear regression models »
Dominik Janzing · Bernhard Schölkopf -
2018 Oral: On Matching Pursuit and Coordinate Descent »
Francesco Locatello · Anant Raj · Sai Praneeth Reddy Karimireddy · Gunnar Ratsch · Bernhard Schölkopf · Sebastian Stich · Martin Jaggi -
2018 Poster: Tempered Adversarial Networks »
Mehdi S. M. Sajjadi · Giambattista Parascandolo · Arash Mehrjou · Bernhard Schölkopf -
2018 Poster: Differentially Private Database Release via Kernel Mean Embeddings »
Matej Balog · Ilya Tolstikhin · Bernhard Schölkopf -
2018 Oral: Differentially Private Database Release via Kernel Mean Embeddings »
Matej Balog · Ilya Tolstikhin · Bernhard Schölkopf -
2018 Oral: Tempered Adversarial Networks »
Mehdi S. M. Sajjadi · Giambattista Parascandolo · Arash Mehrjou · Bernhard Schölkopf -
2018 Poster: Learning Independent Causal Mechanisms »
Giambattista Parascandolo · Niki Kilbertus · Mateo Rojas-Carulla · Bernhard Schölkopf -
2018 Oral: Learning Independent Causal Mechanisms »
Giambattista Parascandolo · Niki Kilbertus · Mateo Rojas-Carulla · Bernhard Schölkopf -
2017 Invited Talk: Causal Learning »
Bernhard Schölkopf