Training AI Co-Scientists Using Rubric Rewards

AI co-scientists are emerging as a useful tool for human researchers, with a crucial ability being proposing a research plan for a given research goal. In this work, we study how to train language models that generate better research plans by leveraging the vast corpus of existing research papers. To collect diverse training data, we automatically extract research goals and goal-specific grading rubrics from papers across domains. We then train models for research plan generation via reinforcement learning, with a frozen copy of the initial policy acting as the grader, using the rubrics to evaluate plans generated by the training policy. To validate this approach, we conduct a human study for machine learning research goals spanning 225 expert hours. The experts prefer plans generated by our finetuned Qwen3-30B-A3B model over the initial model for 70% goals, and over Grok-4-Thinking for 59.6% goals. To assess generality, we also extend our approach to goals from medical papers, and recent arXiv preprints, evaluating with a jury of frontier models. Our finetuning yields 12-22% relative improvements and significant cross-domain generalization, proving effective even in problem settings like medical research where execution feedback is infeasible. Overall, we demonstrate the potential of a scalable training recipe as a step towards improving general AI co-scientists.

AI is now widely used to amplify the productivity of scientists for tasks that are easy to verify. In this work, we study how to train AI that produces research plans, given an open-ended scientific goal. A key challenge is the paucity and cost of expert scientist time needed to evaluate AI generated research plans. It is also challenging, and potentially unsafe to let AI learn via real-world via trial and error, as scientific experiments can be expensive, time-taking, and involve human subjects. We instead propose a way for AI to better learn from the vast corpus of existing research papers. We extract the main research goal from these papers, and also some requirements and features the scientists reported as being important for their proposed solution or experiment plan as grading rubrics. A copy of the AI being trained uses these rubrics to grade the plans generated by the AI, which produces a numerical "reward". This reward can be optimized using popular reinforcement learning algorithms for language models. We show this process leads to improvements in AI generated research plans, with a human study with ML experts for ML research, and automated evaluations with stronger graders across more domains, including medicine.