DRIVE: Best Data Scheduling Practices for Reinforcement Learning with Verifiable Reward in Competitive Code Generation

Recent success of large reasoning models (such as OpenAI o1 and DeepSeek R1) have spurred a resurgence of interest in reinforcement learning from verifiable rewards (RLVR). However, progress is still largely driven by RL algorithm design, while data scheduling -- the data-side decisions that determine what the model trains on over time -- is critical but remains underexplored. Therefore, data scheduling becomes the focus of this paper, including how to curate data for supervised fine-tuning (SFT) and how to select prompts and collect rollouts for reinforcement learning (RL). We introduce a pipeline with careful designs on data scheduling, consisting of hardness-prioritized SFT and two-stage RL. Specifically, we first fine-tune the base model on supervision data that is curated to prioritize difficulty based on both arena learning and classification. Then, we introduce two-stage RL where a decreased max sequence length during rollout is used in the first stage to expand entropy and reduce repetition, and a large number rollouts per prompt and curriculum design are adopted in the second stage to encourage exploration for challenging problems. We implement this pipeline on Qwen2.5-32B and an internal 389B MoE model, and evaluate them on a wide range of benchmarks including challenging LeetCode and Codeforces weekly contests. The results not only indicate the effectiveness and scalability of our pipeline but also demonstrate our model achieve sota of 32B models in competitive code generation.

Recent breakthroughs in AI, such as OpenAI o1 and DeepSeek R1, have shown a remarkable ability to "think" and reason through complex problems. This success has renewed interest in teaching AI through trial-and-error, where the system is rewarded for getting correct answers. While most research focuses on designing better training algorithms, a critical piece of the puzzle is often overlooked: data scheduling, or deciding exactly what lessons the AI should learn and in what order. In this paper, we introduce a novel "curriculum" that optimizes how AI is trained. First, instead of giving the AI random practice problems, we prioritize teaching it the most difficult, logic-heavy concepts first to build a strong foundation. Second, during its trial-and-error training, we initially restrict how much it can write to force it to think concisely and avoid repetitive mistakes, before gradually introducing harder tasks and encouraging broader exploration. We tested our training method on standard-sized and massive AI models. The results show that our approach is highly effective: the trained models achieved state-of-the-art performance in competitive computer programming, excelling at elite, international coding contests like LeetCode and Codeforces. This research proves that how we teach an AI is just as important as the algorithm we use to train it.