Curating the Future: A Scalable Recipe for Training Open-Ended Forecasters

High-stakes decision making involves reasoning under uncertainty about the future. In this work, we train language models to make predictions on open-ended forecasting questions. To scale up training data, we synthesize novel forecasting questions from global events reported in daily news. While directly training on this data leads to performance drops, carefully curating questions creates a valuable training resource. We use the resulting dataset, OpenForesight, to post-train Qwen3 thinking models. To prevent leakage of future information during training and evaluation, we use an offline news corpus, both for data generation and retrieval in our forecasting system. Guided by a small validation set, we show the benefits of retrieval, and an improved reward function for reinforcement learning (RL). Once we obtain our final forecasting system, we perform held-out testing between May to August 2025. Our specialized model, OpenForecaster-8B, matches much larger proprietary models, with our training improving the accuracy, calibration, and consistency of predictions. We find calibration improvements from forecasting training generalize across popular benchmarks. We will open-source our models, code, and data to make LLM based forecasting research broadly accessible.

Good forecasting questions are scarce and mostly skewed towards topics popular in prediction markets. We present a scalable recipe for training language models to forecast open-ended future events beyond these question sets. Our first contribution is an automated pipeline that generates open-ended forecasting questions from daily news. One model explores articles and writes questions about future events, and a second model checks and filters low-quality questions. Applying this to 250,000 articles yields OpenForesight, an open dataset of about 52,000 questions. Our second contribution is showing that careful curation is what makes this data useful, i.e. unfiltered questions often have artifacts, and training on them actually makes models worse by teaching shortcuts; our filtering and rewriting process removes these giveaways. Lastly, we contribute a RL training setup with reward that jointly optimizes accuracy and calibration. Our RL setup avoids future-information leakage by design by using a frozen, dated news archive from commoncrawl. The resulting OpenForecaster-8B model matches much larger proprietary models on both accuracy and calibration, with calibration improvements generalizing to unrelated benchmarks.