Motion Planning in Compressed Representation Spaces

Deep learning methods have vastly expanded the capabilities of motion planning in robotics applications, as learning priors from large-scale data has been shown to be essential in capturing the highly complex behavior required for solving tasks such as manipulation or navigation for autonomous vehicles. At the same time, model-based planning algorithms based on search or optimization remain an essential tool due to their flexibility, efficiency, and the ability to incorporate domain knowledge via expert-designed algorithms and objective functions. We propose a new generative framework to unify these two paradigms. First, we learn an autoencoder with a high compression ratio and a latent space of hierarchically ordered, discrete-valued tokens. Leveraging both the dimensionality reduction and the hierarchical coarse-to-fine structure learned by this autoencoder, we then perform motion planning by directly searching in the latent space of tokens. This search can optimize arbitrary objective functions specified at test time, providing a large degree of flexibility while maintaining efficiency and producing realistic solutions by relying on the generative capabilities of the highly compressed autoencoder. We evaluate our method on nuPlan and the Waymo Open Motion Dataset, showing how latent space search can be used for a variety of guided behavior generation tasks, achieving strong performance for closed-loop motion planning and multi-agent guided scenario synthesis without requiring any task-specific training.

Robots such as self-driving cars need to choose motions that are realistic and appropriate for the situation, while also adapting to changing goals, rules, or user preferences. Traditional planning methods are flexible because they can directly optimize objectives designed by humans, but they often struggle to capture the complex behavior seen in real-world data. Learning-based methods can learn realistic behavior from large datasets, but they are often less flexible when we want to change the desired behavior without training a new model. This paper introduces a way to combine these strengths. We train a model to describe possible future motions using a small set of learned choices. The model takes into account the robot's current surroundings, so these choices change with the situation and correspond to motions that are plausible in the current environment. Given a new objective, such as following a route or performing a lane change, motion planning then becomes a search for the choice that best satisfies that objective. The learned choices are organized from coarse to fine, making this search efficient without having to check every possible motion in detail. Because the model has learned from real-world data, the generated motions remain realistic, while the search procedure makes it easy to specify new objectives. We evaluate this approach on self-driving benchmarks and show that it can generate realistic driving behavior, follow routes in simulation, and produce consistent behavior for multiple interacting agents without task-specific training.