$V_0$: A Generalist Value Model for Any Policy at State Zero

Policy gradient methods rely on a baseline to measure the relative advantage of an action. In Actor-Critic LLM training (\textit{e.g.}, PPO), this baseline is typically estimated by a Value Model as large as the policy. However, as the policy evolves, the value model requires expensive, synchronous training. To avoid this overhead, GRPO eliminates the value model by using the average reward of group rollouts as the baseline; yet, this necessitates extensive sampling to maintain stability. In this paper, we propose $V_0$, a Generalist Value Model capable of estimating the expected performance of any model on unseen prompts without parameter updates. We reframe value estimation by treating the policy's dynamic capability as an explicit context input, leveraging a history of instruction-performance pairs to dynamically profile the model. Acting as a resource scheduler, $V_0$ predicts success rates prior to rollout during GRPO training for efficient sampling budget allocation. During deployment, it functions as a router dispatching instructions to the most cost-effective model. Empirical results demonstrate that $V_0$ significantly outperforms heuristic budget allocation and achieves a Pareto-optimal performance-cost trade-off in LLM routing.

We introduce $V_{0}$, a universal predictor that eliminates the need for retraining entirely. Instead of needing constant updates, $V_{0}$ instantly assesses any AI's current skill level by simply looking at its recent track record. By analyzing this history, $V_{0}$ can accurately predict an AI's expected success rate on a brand new question before it even generates an answer. This breakthrough saves massive computing power during training and efficiently routes everyday user queries to the most cost-effective AI model available.