Benchmarking Physics-Informed Time-Series Models for Operational Global Station Weather Forecasting

The development of Time-Series Forecasting (TSF) models is often constrained by the lack of comprehensive datasets, especially in Global Station Weather Forecasting (GSWF), where existing datasets are small, temporally short, and spatially sparse. To address this, we introduce WEATHER-5K, a large-scale observational weather dataset that better reflects real-world conditions and supports improved model training and evaluation. While recent TSF methods perform well on benchmarks, they still lag behind operational Numerical Weather Prediction (NWP) systems in capturing complex weather dynamics and extreme events. We propose PhysicsFormer, a physics-informed forecasting model that combines a dynamic core with a Transformer residual to predict future weather states. Physical consistency is enforced via pressure–wind alignment and energy-aware smoothness losses, ensuring plausible dynamics while capturing complex temporal patterns. We benchmark PhysicsFormer and other TSF models against operational systems across several weather variables, extreme event prediction, and model complexity, providing a comprehensive assessment of the gap between academic TSF models and operational forecasting. The dataset and benchmark implementation are available at: https://github.com/taohan10200/WEATHER-5K.

Weather forecasts affect everyday decisions and are especially important for preparing for heatwaves, storms, and other extreme events. This paper studies how artificial intelligence can be used to forecast weather at observation stations around the world. A key challenge is that many existing research datasets are too small or too limited to reflect real-world weather conditions. To address this, we introduce WEATHER-5K, a large dataset built from 10 years of hourly observations collected at 5,672 weather stations worldwide. Using this dataset, we compare many recent AI forecasting methods with professional weather forecasting systems used in practice. We find that although AI methods are efficient and promising, they still fall behind operational systems, especially for longer forecasts and extreme weather events. We also propose PhysicsFormer, a new model that combines data-driven learning with basic physical relationships in the atmosphere. Our dataset, benchmark, and findings provide a clearer path for developing more reliable and practical AI-based weather forecasting methods.