BRIDGE: Predicting Human Task Completion Time From Model Performance

Evaluating the real-world capabilities of AI systems requires grounding benchmark performance in human-interpretable measures of task difficulty. Existing approaches that rely on direct human task completion time annotations are costly, noisy, and difficult to scale across benchmarks. In this work, we propose BRIDGE, a unified psychometric framework that learns the latent difficulty scale from model responses and anchors it to human task completion time. Using a two-parameter logistic Item Response Theory model, we jointly estimate latent task difficulty and model capability from model performance data across multiple benchmarks. We demonstrate that latent task difficulty varies linearly with the logarithm of human completion time, allowing human task completion time to be inferred for new benchmarks from model performance alone. Leveraging this alignment, we forecast frontier model capabilities in terms of human task length and independently reproduce METR’s exponential scaling results, with the 50% solvable task horizon doubling approximately every 6 months.

AI systems are usually evaluated with benchmark scores, but these scores can be hard to interpret: a higher score does not clearly tell us what kinds of real-world tasks an AI system can actually complete, or how much human effort those tasks would require. A more understandable measure is time: for example, whether an AI system can complete tasks that would take a person minutes, hours, or longer. In this work, we introduce BRIDGE, a method that connects AI benchmark performance to estimated human task completion time. Instead of asking annotators to time themselves on every new benchmark task, BRIDGE learns from patterns of which models succeed or fail on which tasks. It then uses a small set of tasks with known human completion times to translate this learned difficulty scale into human time. We find that this approach can accurately estimate task duration across several challenging benchmarks, including software engineering, cybersecurity, and machine learning tasks. BRIDGE also allows us to track AI progress over time in human-understandable units, showing that the length of tasks frontier models can solve has been growing rapidly, roughly doubling every six months.