Improve Reasoning Ability by Reinforcing Only from Positive Rollouts

Reinforcement learning with verifiable rewards (RLVR) becomes a dominant paradigm for enhancing the reasoning abilities of large language models (LLMs). However, we note that under sparse binary rewards, negative rollouts admit no gradation of failure severity, and the combinatorial vastness makes penalizing a few sampled negatives unlikely to cover a meaningful gradient signal. In this work, we propose Positive-Only Policy Optimization (POPO), a novel RLVR framework in which learning exclusively from positive rollouts via bounded importance sampling over the positive set. We show that implicit negative gradients can emerge naturally through reinforcing the positive probability via rollouts redistribution. Next, POPO stabilizes policy optimization through a momentum-updated siamese network and a bounded representation space similarity penalty that replaces the KL-divergence. We conduct experiments using public text LLMs across mathematical benchmarks and find that POPO matches or even surpasses GRPO. Notably, we show that POPO can achieve 36.67% in AIME 2025 with Qwen-Math-7B, outperforming GRPO 30.00%. Our ablation and sweep studies further illustrate the necessity and robustness of POPO components.