Scaling Laws of Global Weather Models

Data-driven models are revolutionizing weather forecasting. To optimize training efficiency and model performance, this paper analyzes empirical scaling laws within this domain. We investigate the relationship between model performance (validation loss) and three key factors: model size ($N$), dataset size ($D$), and compute budget ($C$). Across a range of models, we find that Aurora exhibits the strongest data-scaling behavior: increasing the training dataset by 10× reduces validation loss by up to 3.2×. GraphCast demonstrates the highest parameter efficiency, yet suffers from limited hardware utilization. Our compute-optimal analysis indicates that, under fixed compute budgets, allocating resources to more total training data yields greater performance gains than increasing model size. Furthermore, we analyze model shape and uncover scaling behaviors that differ fundamentally from those observed in language models: weather forecasting models consistently favor increased width over depth. These findings suggest that future weather models should prioritize wider architectures and larger effective training datasets to maximize predictive performance.

Artificial intelligence is currently revolutionizing how we predict the weather, but figuring out the most efficient way to train these massive programs is a major challenge. We wanted to understand the "scaling laws" of weather AI—meaning how a system's forecasting accuracy improves when we increase its size, the amount of data it learns from, or the computing power used to train it. By comparing several leading AI weather models, we discovered that these systems actually learn fundamentally differently than popular language models. Specifically, weather models become much more accurate when their internal mathematical networks are built "wider" rather than "deeper". We also found that if researchers have a limited computing budget, it is far better to feed a model more historical weather data than it is to simply make the model itself larger. Ultimately, these insights provide a clear blueprint for developers, showing that future weather-predicting AI should prioritize wider designs and massive training datasets to give us the most accurate and computationally efficient forecasts possible.