Scaling Agentic Verifier for Competitive Coding

Large language models (LLMs) have demonstrated strong coding capabilities but still struggle to solve competitive programming problems correctly in a single attempt. Execution-based re-ranking offers a promising test-time scaling strategy, yet existing methods are constrained by either difficult test case generation or inefficient random input sampling. To address this limitation, we propose **Agentic Verifier**, an execution-based agent that actively reasons about program behaviors and searches for highly discriminative test inputs that expose behavioral discrepancies among candidate solutions. Through multi-turn interaction with code execution environments, the verifier iteratively refines the candidate input generator and produces targeted counterexamples rather than blindly sampling inputs. We train the verifier to acquire this discriminative input generation capability via a scalable pipeline combining large-scale data synthesis, rejection fine-tuning, and agentic reinforcement learning. Extensive experiments across five competitive programming benchmarks demonstrate consistent improvements over strong execution-based baselines, achieving up to **+10-15\%** absolute gains in Best@$k$ accuracy. Further analysis reveals clear test-time scaling behavior and highlights the verifier’s broader potential beyond reranking.

Large language models can now write computer programs, but they still often make mistakes on difficult coding problems. Existing methods usually test programs using random inputs, but random tests rarely uncover the hidden bugs that distinguish correct solutions from incorrect ones. We introduce Agentic Verifier, a system that actively searches for more informative test cases. Instead of testing programs blindly, it analyzes how different solutions behave and iteratively designs new inputs that are more likely to reveal errors. Across five competitive programming benchmarks, our method consistently identifies correct programs more accurately than existing approaches, improving performance by up to 10–15%. More broadly, our work suggests that AI coding systems can become more reliable by actively searching for challenging corner cases rather than relying on fixed or random tests.