Invited Talk - Open Training Recipes for Mathematical Reasoning in Language Models

Mathematical reasoning remains a challenge for large language models, requiring systematic approaches to integrate reasoning capabilities throughout the training pipeline. In this talk I will discuss the open recipes we developed to improve OLMo and Tülu’s reasoning capabilities, ranging from synthetic data generation, pre-training/mid-training/post-training techniques, and reinforcement learning from verifiable rewards (RLVR). I will put a special focus on how we innovate RLVR for math. Finally, I'll be honest about what works and what doesn't when it comes to open approaches for math reasoning in LLMs—the good, the bad, and the limitations we're still trying to figure out.