NavOL: Navigation Policy with Online Imitation Learning

Learning robust navigation policies remains a core challenge in robotics. Offline imitation learning suffers from distribution shift and compounding errors at rollout, while reinforcement learning requires reward engineering and learns inefficiently. In this paper, we propose NavOL, an online imitation learning paradigm that interacts with a simulator and updates itself using expert demonstrations gathered online. Built upon a pretrained navigation diffusion policy that maps local observations to future waypoints, NavOL trains in a rollout–update loop: during rollout, the policy acts in the simulator and queries a global planner which has privileged access to the global environment for the optimal path segment as ground truth trajectory labels; during update, the policy is trained on the online collected observation–trajectory pairs. This online imitation loop removes the need for reward design, improves learning efficiency, and mitigates distribution shift by training on the policy’s own explored rollouts. Built on IsaacLab with fast, high-fidelity parallel rendering and domain randomization of camera pose and start-goal pairs, our system scales across 50 scenes on 8 RTX 4090 GPUs, collecting over 2,000 new trajectories per hour, each averaging more than 400 steps. We also introduce an indoor visual navigation benchmark with predefined start and goal positions for zero-shot generalization. Extensive evaluations on simulation benchmarks, including the NavDP benchmark and our proposed benchmark, as well as carefully designed real-world experiments, demonstrate the effectiveness of NavOL, showing consistent performance gains in online imitation learning.

Teaching a robot to find its way through a home or office it has never seen before is surprisingly hard. One common approach is to let the robot copy recorded examples of good navigation, but a robot trained this way tends to wander into unfamiliar situations and make small mistakes that snowball into getting stuck or lost. Another approach lets the robot learn by trial and error, but this needs carefully hand-designed scoring rules and takes a very long time to learn. We introduce NavOL, a method that teaches a navigation robot inside a highly realistic computer simulation. As the robot moves around and explores on its own, an expert "guide" that can see the full map of the environment continuously shows it the best route from wherever it currently is to its goal. The robot keeps practicing and improving from these on-the-spot corrections, so it learns directly from the very situations it actually runs into rather than from a fixed set of recorded examples. This avoids the snowballing-mistake problem and removes the need for hand-tuned scoring rules. Because the simulation runs many virtual environments at once, the system learns extremely fast: using 8 standard graphics cards across 50 different indoor scenes, it gathers more than 2,000 fresh practice routes every hour. We also build a new test collection of indoor environments the robot has never trained on, to fairly measure how well it handles new places. In both simulated tests and real-world robot experiments, NavOL navigates more reliably than existing methods, showing that letting a robot learn from its own exploration with live expert guidance is a powerful and efficient way to teach navigation.