(Oral) Agentic Forecasting using Sequential Bayesian Updating of Linguistic Beliefs
Abstract
We present the Bayesian Linguistic Forecaster (BLF), an agentic system for binary forecasting that achieves state-of-the-art performance on the ForecastBench benchmark. The system is built on three ideas: (1) a \emph{linguistic belief state} that combines a numeric probability with natural-language evidence summaries and is updated by the LLM at each step of an iterative tool-use loop; (2) \emph{multi-trial aggregation} via shrunken logit-space averaging; and (3) \emph{hierarchical Platt calibration} with per-source intercepts, which avoids over-shrinking extreme predictions on sources with skewed base rates. On 400 questions from the ForecastBench leaderboard, BLF outperforms all top public methods, including Cassi, GPT-5, Grok 4.20, and Foresight-32B, and is the only method to significantly beat a strong LLM-free crowd+empirical-prior baseline. Mixed-effects ablations across five base LLMs show that all three components contribute, with their relative importance depending on the base model and on whether a crowd anchor is available.