R2-Router: A New Paradigm for LLM Routing with Reasoning

As LLMs proliferate with diverse capabilities and costs, LLM routing has emerged by learning to predict each LLM's quality and cost for a given query, then selecting the one with high quality and low cost. However, existing routers implicitly assume a single fixed quality and cost per LLM for each query, ignoring that the same LLM's quality varies with its output length. This causes routers to exclude powerful LLMs when their estimated cost exceeds the budget, missing the opportunity that these LLMs could still deliver high quality at reduced cost with shorter outputs. To address this, we introduce R2-Router, which treats output length budget as a controllable variable and jointly selects the best LLM and length budget, enforcing the budget via length-constrained instructions. This enables R2-Router to discover that a powerful LLM with constrained output can outperform a weaker LLM at comparable cost—efficient configurations invisible to prior methods. Together with the router framework, we construct R2-Bench, the first routing dataset capturing LLM behavior across diverse output length budgets. Experiments show that R2-Router achieves state-of-the-art performance at $4-5\times$ lower cost compared with existing routers. This work opens a new direction: routing as reasoning, where routers evolve from reactive selectors to deliberate reasoners that explore which LLM to use and at what cost budget. The code is publicly available at https://github.com/UCF-ML-Research/R2-Router.

Modern AI offers many language models, from small and cheap to large and expensive. "Router" tools automatically pick a model for each question to get good answers at low cost, but today's routers assume every model has a single fixed cost, so they often discard powerful models as too expensive. We observed that a model's answer quality and its cost both depend on how long its answer is, and that this length can be controlled simply by telling the model how many words to use. Our method, R2-Router, treats answer length as an extra dial it can turn: instead of only choosing which model to use, it also chooses how long that model's answer should be. This reveals efficient options others miss—for example, a strong model giving a brief answer can beat a weak model at the same cost. We also built R2-Bench, the first dataset recording how each model's quality changes as its answers grow longer or shorter. In experiments, R2-Router matches existing routers' quality at one-fourth to one-fifth of the cost, and ranks first on a public router-comparison leaderboard. By lowering the cost of high-quality AI, this work helps make advanced AI more affordable and energy-efficient.