Timezone: »

 
Poster
Invariant Causal Prediction for Block MDPs
Amy Zhang · Clare Lyle · Shagun Sodhani · Angelos Filos · Marta Kwiatkowska · Joelle Pineau · Yarin Gal · Doina Precup

Tue Jul 14 03:00 PM -- 03:45 PM & Wed Jul 15 04:00 AM -- 04:45 AM (PDT) @
in Poster Session 9 »

Generalization across environments is critical to the successful application of reinforcement learning (RL) algorithms to real-world challenges. In this work we propose a method for learning state abstractions which generalize to novel observation distributions in the multi-environment RL setting. We prove that for certain classes of environments, this approach outputs, with high probability, a state abstraction corresponding to the causal feature set with respect to the return. We give empirical evidence that analogous methods for the nonlinear setting can also attain improved generalization over single- and multi-task baselines. Lastly, we provide bounds on model generalization error in the multi-environment setting, in the process showing a connection between causal variable identification and the state abstraction framework for MDPs.

Keywords: Causality · Representation Learning · Reinforcement Learning · Reinforcement Learning Theory · Reinforcement Learning - General

Author Information

Amy Zhang (McGill University)
Clare Lyle (University of Oxford)
Shagun Sodhani (Facebook AI Research)
Angelos Filos (University of Oxford)
Marta Kwiatkowska (Oxford University)
Joelle Pineau (McGill University / Facebook)
Yarin Gal (University of Oxford)
Doina Precup (McGill University / DeepMind)

More from the Same Authors