JURY-RL: Votes Propose, Proofs Dispose for Label-Free RLVR

Reinforcement learning with verifiable rewards (RLVR) enhances reasoning in large language models (LLMs) but typically depends on human-annotated answers or curated reward specifications. Label-free alternatives such as majority voting or LLM-as-a-Judge remove this cost but introduce false positives that destabilize training. We introduce JURY-RL, a label-free RLVR framework that decouples answer proposal from reward disposal: rollouts matching the plurality-voted answer receive positive reward only when that answer is verified in Lean. When verification is inconclusive, ResZero (Residual-Zero) discards the unverified plurality and redistributes a zero-mean, variance-preserving signal over the residual answers, keeping the optimization gradient stable without reinforcing unverifiable consensus. Across three backbones trained on math data, JURY-RL outperforms label-free baselines on mathematical reasoning, transfers to code generation and general benchmarks, and surpasses a majority-vote baseline in the test-time RL (TTRL) setting; it matches supervised ground-truth training on pass@1 and beats it on pass@k and response diversity.