Timezone: »

 
Poster
Efficient Reinforcement Learning in Block MDPs: A Model-free Representation Learning approach
Xuezhou Zhang · Yuda Song · Masatoshi Uehara · Mengdi Wang · Alekh Agarwal · Wen Sun

Thu Jul 21 03:00 PM -- 05:00 PM (PDT) @ Hall E #1413
in Poster Session 3 »

We present BRIEE, an algorithm for efficient reinforcement learning in Markov Decision Processes with block-structured dynamics (i.e., Block MDPs), where rich observations are generated from a set of unknown latent states. BRIEE interleaves latent states discovery, exploration, and exploitation together, and can provably learn a near-optimal policy with sample complexityscaling polynomially in the number of latent states, actions, and the time horizon, with no dependence on the size of the potentially infinite observation space.Empirically, we show that BRIEE is more sample efficient than the state-of-art Block MDP algorithm HOMER and other empirical RL baselines on challenging rich-observation combination lock problems which require deep exploration.

Keywords: RL: Function Approximation · T: Reinforcement Learning and Planning

Author Information

Xuezhou Zhang (University of Wisconsin-Madison)
Yuda Song (Carnegie Mellon University)
Masatoshi Uehara (Cornell University)
Mengdi Wang (Princeton University)
Alekh Agarwal (Microsoft Research)
Wen Sun (Cornell University)

Related Events (a corresponding poster, oral, or spotlight)

More from the Same Authors