Laplacian Representations for Decision-Time Planning

Planning with a learned model remains a key challenge in model-based reinforcement learning (RL). In decision-time planning, state representations are critical as they must support local cost computation while preserving long-horizon structure. In this paper, we show that the Laplacian representation provides an effective latent space for planning by capturing state-space distances at multiple time scales. This representation preserves meaningful distances and naturally decomposes long-horizon problems into subgoals, also mitigating the compounding errors that arise over long prediction horizons. Building on these properties, we introduce ALPS, a hierarchical planning algorithm, and demonstrate that it outperforms commonly used model-free baselines on a selection of offline goal-conditioned RL tasks from OGBench.

How can a robot learn to navigate from any starting point to a goal, using only pre-collected data and without further interaction with the environment? We address this challenge by learning a compact representation of the world that enables long-horizon planning. We propose using the Laplacian representation of the state space which spatially clusters states that are temporally close in the dataset. This representation thus has a powerful property: distances between states directly reflect how hard it is to travel between them -- states that are easy to reach are nearby, and states separated by obstacles or long paths are far apart. This makes it naturally suited for planning. Building on this, we introduce ALPS (Augmented Laplacian Planning with Subgoals), a hierarchical planning algorithm, which first clusters the representation space to identify a set of subgoals defined as intermediate waypoints that break the long-horizon problem into manageable steps. It then plans a high-level path through these subgoals and executes low-level actions to reach each one in sequence. We evaluate ALPS on a suite of challenging offline robot navigation and manipulation tasks, demonstrating the practical value of Laplacian representation for planning.