Bottleneck-Guided Spectral Subgoals For Offline Goal-Conditioned RL

Offline goal-conditioned RL (OGCRL) learns to reach arbitrary goals from the offline dataset, but long-horizon performance hinges on crossing a handful of hard-to-cross bottlenecks. These bottlenecks not only dictate the feasible paths toward the goal but also act as critical keypoints, marking the transitions between adjacent regions and providing the agent with essential directional guidance. Prior hierarchical methods pick subgoals by time or short-horizon value heuristics, which do not localize the bottleneck, and therefore the agent loses the clear guidance that bottlenecks could provide about where to pass next. We instead model long-horizon planning as ``cross the next bottleneck'': we apply Laplacian spectral clustering to the offline dataset to expose bottlenecks, identify trajectories from the offline dataset that cross these boundaries, and use the corresponding boundary-supported states as keypoint (KP) candidates. The most representative KPs are then instantiated from these candidates, and a directed KP reachability graph $\mathcal G_{\mathrm{KP}}$ is constructed based on the resulting KPs. We then restrict high-level choices to these bottleneck states and use a pluggable low-level controller to execute the short transitions between them. We provide theory showing that under a standard metastable decomposition of the state space, routing through bottlenecks yields an (approximately) optimal one-step subgoal in terms of hitting-time, and that Laplacian spectra recover bottlenecks with high overlap. Thus, Laplacian spectral clustering can discover near-optimal subgoals under these assumptions. Empirically, the same pattern holds: across D4RL and OGBench, our method achieves state-of-the-art results on a broad set of navigation and manipulation tasks and across diverse dataset regimes, for example, **96.5\%** on **AntMaze** and **84.7\%** on **Franka-Kitchen**.

Many artificial intelligence agents need to complete tasks that require many steps, such as moving through a maze or manipulating objects in a kitchen. A common difficulty is that success often depends on passing through a few critical places, such as narrow corridors, doorways, or important transition points. If the agent does not recognize these places, it may waste time moving locally or choose unhelpful intermediate targets. This paper proposes a method that helps an agent identify and use these critical transition points. Instead of asking the agent to choose subgoals based only on time or short-term scores, our method first analyzes the offline data to discover the main regions of the environment and the boundaries between them. The states near these boundaries are treated as keypoints, because they often indicate where the agent must pass in order to make real progress toward the final goal. After finding these keypoints, the agent builds a simple graph that describes which keypoints can be reached from others. At test time, it plans a route through this graph and then uses a low-level controller to move from one keypoint to the next. This separates long-horizon planning from short local control, making the overall task easier. Our theoretical analysis explains why bottleneck keypoints can serve as useful intermediate goals under suitable assumptions, and our experiments show that this idea works well across a range of navigation and manipulation tasks. Overall, the method helps offline agents make better long-term decisions by focusing on the important transition points that structure the environment.