Learn Hard Problems During RL with Reference Guided Fine-tuning

Reinforcement learning (RL) has become a central mechanism for improving mathematical reasoning in large language models, yet its effectiveness is often limited by reward sparsity: for challenging problems, models fail to sample correct reasoning trajectories, leading to stalled or inefficient learning. We introduce Reference-Guided Fine-Tuning (ReGFT), a pre-RL initialization strategy that leverages reference solutions to improve the effectiveness of subsequent RL. ReGFT uses partial reference guidance to elicit model-generated reasoning traces that remain aligned with the model’s own inference distribution, enabling RL to operate in regimes where reward signals would otherwise be absent. Across three challenging benchmarks (AIME 2024, AIME 2025, and Beyond-AIME), models initialized with ReGFT consistently achieve faster RL convergence, higher final accuracy, and stronger inference-time scaling compared to both raw and ReFT baseline. These results demonstrate that ReGFT can substantially enhance the effectiveness of reinforcement learning for mathematical reasoning.