DriveGPT: Scaling Autoregressive Behavior Models for Driving

We present DriveGPT, a scalable behavior model for autonomous driving. We model driving as a sequential decision-making task, and learn a transformer model to predict future agent states as tokens in an autoregressive fashion. We scale up our model parameters and training data by multiple orders of magnitude, enabling us to explore the scaling properties in terms of dataset size, model parameters, and compute. We evaluate DriveGPT across different scales in a planning task, through both quantitative metrics and qualitative examples, including closed-loop driving in complex real-world scenarios. In a separate prediction task, DriveGPT outperforms state-of-the-art baselines and exhibits improved performance by pretraining on a large-scale dataset, further validating the benefits of data scaling.

• Our paper is the first published study of large-scale scaling laws in autonomous driving. The largest model was trained on 100M+ high-quality human demonstrations in dense urban driving with 1B+ parameters.* Our scaling experiments validate the benefits of increasing both data and compute, revealing better model scalability as training data increases -- consistent with trends observed in language models.* We quantitatively and qualitatively evaluate models across scales, including real-world deployment of our model as a real-time planner in complex urban scenarios.* Our model achieves state-of-the-art performance on the Waymo Open Motion Dataset across key geometric metrics, highlighting its strong generalization in motion prediction.