The Signal is in the Steps: Local Scoring for Reasoning Data Selection

Distilling long-form reasoning from teacher models into smaller students requires selecting which candidate solutions to train on. Recent work argues that one should select responses the student model assigns highest probability, i.e., favoring solutions ``natural'' to the student. However, we find that this approach works within a single teacher but fails when scaling to long reasoning traces from multiple diverse teachers. We identify a key cause: this approach scores entire solutions, but students generalize by recombining familiar reasoning steps, not by memorizing complete solutions. Full-trajectory scoring optimizes the wrong target; it rewards global fluency while the transferable signal lies in local step transitions. We propose Local Average Log Probability (LALP), which scores each reasoning step using only a small window of preceding context, measuring whether each step is justified by its immediate premises rather than whether the full response looks natural to the student. LALP enables two practical use cases: selecting the best teacher before fine-tuning and curating training data from diverse teacher pools. Across math, coding, and science reasoning tasks, LALP consistently improves accuracy when selecting the most natural solutions by a large margin.

Imagine two correct solutions to the same math problem. One uses an advanced method that only an expert might appreciate, while another explains every step like a beginner-friendly tutor. Both are correct, but different students may learn better from different explanations. The same problem appears when training AI models. Different AI teachers can produce multiple correct explanations for the same question, and we need to choose which one an AI student should learn from. One common approach is to pick the explanation that sounds most fluent to the student model overall. However, we show that this can be misleading for long reasoning. We observe that a polished answer may simply repeat familiar phrases or follow a familiar style, while the important logic is buried inside the text. Our method looks at explanations step by step. Instead of asking, “Does the whole answer sound good?”, it asks, “Does each step clearly follow from the nearby, local steps before it?” This helps select explanations with reasoning steps that are easier for the student model to learn and reuse. In experiments, this step by step selection method helped AI student models learn more effectively from stronger AI teachers, improving math accuracy by nearly 10 percentage points.