V1: Unifying Generation and Self-Verification for Parallel Reasoners

Test-time scaling for complex reasoning tasks shows that leveraging inference-time compute, by methods such as independently sampling and aggregating multiple solutions, results in significantly better task outcomes. However, a critical bottleneck is verification: sampling is only effective if correct solutions can be reliably identified among candidates. While existing approaches typically evaluate candidates independently via scalar scoring, we demonstrate that models are substantially stronger at pairwise verification. Leveraging this insight, we introduce V₁, a framework that unifies generation and verification through efficient pairwise ranking. V₁ comprises two components: V₁-Infer, an uncertainty-guided algorithm using a tournament-based ranking that dynamically allocates self-verification compute to candidate pairs whose relative correctness is most uncertain; and V₁-PairRL, an RL framework that jointly trains a single model as both generator and pairwise self-verifier, ensuring the verifier adapts to the generator's evolving distribution. On competitive code generation, SWE and terminal-based agents, math reasoning, and medical QA benchmarks, V₁-Infer improves Pass@1 by up to 10% over pointwise verification and outperforms SoTA aggregation-based test-time scaling while being more efficient. Furthermore, V₁-PairRL achieves test-time scaling gains of 3.8–10pp over standard RL and pointwise joint training on code, math, and non-verifiable domains, and improves base Pass@1 by up to 8.7pp over standard RL. Project page & code.

Modern AI (LLM) systems can often improve at hard reasoning by generating many candidate answers and picking the best — but picking is the hard part. When an AI grades (judges) each attempt alone, the scores are unreliable, and a confidently stated wrong answer often wins. Our work takes a different approach: the model compares attempts head-to-head, like matchups in a chess tournament. V1-Infer, our algorithm, pairs candidates and spends extra effort on the closest contests, where the model is least sure which side wins. V1-PairRL, our training recipe, builds a single AI to be both a strong problem-solver and a strong head-to-head judge of its own answers, so it is trained for two skills together. The intuition is that if we train the AI on these two skills, we can use the same AI model to generate many answers, as well as judge its own answers, better than an AI that is not trained to judge its own answers. We tested our method for verifying answers from LLMs and Agents on tasks such as writing code for programming contests, fixing real bugs in popular software repositories, executing tasks in a Linux terminal, solving math problems, and answering open-ended medical questions. Our method significantly improves the judgment capability of LLMs compared with the standard "grade each attempt alone" approach, and also leads to better answers compared to methods that have the AI read its own prior attempts and produce a new answer (technically termed as self-aggregation-based or self-refinement-based methods). Even when the head-to-head AI judge is a smaller AI than the answer-writer, most of the gain holds.