LiveOIBench: Can Large Language Models Outperform Human Contestants in Informatics Olympiads?

Competitive programming problems are increasingly used to evaluate the coding capabilities of large language models (LLMs) due to their complexity and ease of verification. Yet, current coding benchmarks face limitations such as a lack of exceptionally challenging problems, insufficient test case coverage, and reliance on online platform APIs that limit accessibility. To address these issues, we introduce LiveOIBench, a large-scale competitive programming benchmark featuring $403$ expert-curated problems, averaging $60$ official test cases each, drawn from 72 contests across 14 Informatics Olympiads held between 2023 and 2025. LiveOIBench has four key features: (1) expert-designed tasks with detailed subtask rubrics and extensive test cases; (2) direct comparison to elite human contestants; (3) continuous updates to reduce contamination risk; and (4) a fully offline, reproducible evaluation system. Benchmarking $34$ popular general-purpose and reasoning LLMs, we find that GPT-5 achieves an 81.76th percentile, still falling short of top human contestants, while among the open-weight models, GPT-OSS-120B reaches only the 60th percentile. Reasoning-trace analyses indicate that robust reasoning models prioritize precise problem analysis over excessive exploration. Finally, analyses across release dates, task familiarity, and code similarity find minimal evidence of data contamination in our benchmark. Our leaderboard, code, and data are available at: https://liveoibench.github.io/.

Modern AI systems are getting better at writing code, but it is hard to know whether they can solve truly difficult programming problems rather than succeed on benchmarks with weak tests or leaked examples. We introduce LiveOIBench, a new benchmark built from recent Informatics Olympiad contests, where top students solve challenging programming problems under strict time and memory limits. Unlike many coding benchmarks, LiveOIBench uses official contest problems, carefully designed test cases, detailed partial-credit scoring, and real contestant results, so AI models can be compared directly with skilled human programmers. We evaluated 34 leading AI models, including both commercial and publicly released systems. The best model, GPT-5, outperformed most contestants on average, but still did not reach the level of top medal-winning humans. We also found that models struggle most when problems require a clever new plan, especially when the solution depends on breaking a task into many connected subproblems. Our analysis suggests that stronger models do better not by exploring many random ideas, but by spending more effort on focused problem analysis and verification. LiveOIBench provides a public, reproducible way to measure progress toward expert-level AI coding.