CoIRL-AD: Collaborative-Competitive Imitation-Reinforcement Learning in Latent World Models for Autonomous Driving

End-to-end autonomous driving models trained with imitation learning (IL) often generalize poorly, particularly in long-tail scenarios where expert demonstrations are sparse. Reinforcement learning (RL) can provide complementary task-level supervision, but applying RL to real-world autonomous driving is challenging in offline settings without interactive simulators, where datasets are dominated by expert actions and provide limited behavioral diversity. We propose CoIRL-AD, a competitive dual-policy framework that integrates IL and RL under a unified offline training regime. CoIRL-AD decouples imitation and reward optimization into separate actors to alleviate objective conflicts, uses imagined future rollouts for long-horizon reward estimation, and introduces a competition mechanism that selectively transfers beneficial behaviors while keeping RL anchored to expert-like driving. Experiments on the nuScenes benchmark show that CoIRL-AD consistently improves robustness over strong IL-based baselines, with especially large gains in cross-city generalization and long-tail scenarios. Code is available at: https://github.com/SEU-zxj/CoIRL-AD.

Autonomous driving systems are often trained by imitation learning, where the model learns to drive by copying human drivers. However, simply copying examples may not prepare a vehicle for rare or difficult situations. Similar to how a good student learns not only by following a teacher but also by solving problems independently, autonomous driving systems can also benefit from reinforcement learning, where they improve through trial and error. Applying reinforcement learning to real-world driving is difficult because it is unsafe and impractical to let experimental systems freely learn on public roads. To address this challenge, we train a “world model” that simulates future driving situations using recorded driving data. This allows the autonomous driving system to practice decision-making safely in a virtual environment. Our method combines imitation learning and reinforcement learning within a unified framework. Experiments show that this combination produces more robust driving behavior than imitation learning alone, especially in challenging scenarios and in new cities that were not seen during training.