Timezone: »

 
Poster
Reachability-Aware Laplacian Representation in Reinforcement Learning
Kaixin Wang · Kuangqi Zhou · Jiashi Feng · Bryan Hooi · Xinchao Wang

Wed Jul 26 05:00 PM -- 06:30 PM (PDT) @ Exhibit Hall 1 #120
in Poster Session 4 »

In Reinforcement Learning (RL), Laplacian Representation (LapRep) is a task-agnostic state representation that encodes the geometry of the environment. A desirable property of LapRep stated in prior works is that the Euclidean distance in the LapRep space roughly reflects the reachability between states, which motivates the usage of this distance for reward shaping. However, we find that LapRep does not necessarily have this property in general: two states having a small distance under LapRep can actually be far away in the environment. Such a mismatch would impede the learning process in reward shaping. To fix this issue, we introduce a Reachability-Aware Laplacian Representation (RA-LapRep), by properly scaling each dimension of LapRep. Despite the simplicity, we demonstrate that RA-LapRep can better capture the inter-state reachability as compared to LapRep, through both theoretical explanations and experimental results. Additionally, we show that this improvement yields a significant boost in reward shaping performance and benefits bottleneck state discovery.

[ PDF]  [ Poster] 

Author Information

Kaixin Wang (Technion)
Kuangqi Zhou (National University of Singapore)
Jiashi Feng (ByteDance)
Bryan Hooi (National University of Singapore)
Xinchao Wang (National University of Singapore)

More from the Same Authors