Timezone: »

 
Poster
Predictive Coding for Locally-Linear Control
Rui Shu · Tung Nguyen · Yinlam Chow · Tuan Pham · Khoat Than · Mohammad Ghavamzadeh · Stefano Ermon · Hung Bui

Thu Jul 16 12:00 PM -- 12:45 PM & Fri Jul 17 01:00 AM -- 01:45 AM (PDT) @ None #None
in Poster Session 45 »

High-dimensional observations and unknown dynamics are major challenges when applying optimal control to many real-world decision making tasks. The Learning Controllable Embedding (LCE) framework addresses these challenges by embedding the observations into a lower dimensional latent space, estimating the latent dynamics, and then performing control directly in the latent space. To ensure the learned latent dynamics are predictive of next-observations, all existing LCE approaches decode back into the observation space and explicitly perform next-observation prediction---a challenging high-dimensional task that furthermore introduces a large number of nuisance parameters (i.e., the decoder) which are discarded during control. In this paper, we propose a novel information-theoretic LCE approach and show theoretically that explicit next-observation prediction can be replaced with predictive coding. We then use predictive coding to develop a decoder-free LCE model whose latent dynamics are amenable to locally-linear control. Extensive experiments on benchmark tasks show that our model reliably learns a controllable latent space that leads to superior performance when compared with state-of-the-art LCE baselines.

Keywords: Deep Generative Models · Deep Reinforcement Learning · Planning and Control · Representation Learning

Author Information

Rui Shu (Stanford University)
Tung Nguyen (VinAI Research, Vietnam)
Yinlam Chow (Google)
Tuan Pham (VinAI Research)
Khoat Than (VinAI & HUST)
Mohammad Ghavamzadeh (Google Research)
Stefano Ermon (Stanford University)
Hung Bui (VinAI Research)

More from the Same Authors