BlitzRank: Principled Zero-shot Ranking Agents with Tournament Graphs

Selecting the top $m$ from $n$ items via expensive $k$-wise comparisons is central to settings ranging from LLM-based document reranking to crowdsourced evaluation and tournament design. Existing methods either rely on heuristics that discard comparison information, or exploit it at prohibitive cost. We introduce a *tournament graph* framework that provides a principled foundation for $k$-wise ranking. Our key observation is that each $k$-item comparison reveals an induced tournament of ${k \choose 2}$ pairwise preferences; aggregating these into a global preference graph and computing its transitive closure yields many additional orderings without further oracle calls. We formalize when the current top-$m$ output is *certifiably determined* and design a greedy query schedule that maximizes information gain towards identifying the top-$m$ items. The framework also gracefully handles non-transitive preferences -- cycles induced by real-world oracles -- by collapsing them into equivalence classes that yield principled *tiered rankings*. Applied to LLM reranking across 14 benchmarks and 5 models, BlitzRank achieves Pareto dominance over existing approaches: matching or exceeding accuracy while requiring 25--40\% fewer tokens than comparable methods; against pairwise reranking, it achieves near-identical quality with 7$\times$ fewer tokens. Code available at [https://github.com/ContextualAI/BlitzRank](https://github.com/ContextualAI/BlitzRank).

Selecting the top items from a pool using multi-item comparisons is important for search engines, human preference evaluations, and tournament design. These comparisons rely on large language models (LLMs) or human experts, which are expensive to query. However, existing methods discard information about non-winning items, leading to high computational costs. We introduce BlitzRank, a framework based on tournament graphs. When comparing a group of items, BlitzRank retains all relationships within that group to eliminate redundant queries. It also handles circular preferences (where A beats B, B beats C, and C beats A) by grouping ambiguous items into relevance tiers rather than dismissing them as noise. Across 14 benchmarks, BlitzRank offers the best accuracy-to-cost tradeoff and makes AI ranking systems faster and cheaper to deploy.