Single-Rollout Hidden-State Dynamics for Training-Free RLVR Data Selection

Reinforcement learning with verifiable rewards (RLVR) can yield large reasoning gains from very few training instances, yet its strong sensitivity to which instances are used makes data selection a central bottleneck. Most existing selection pipelines rely on training-time optimization signals and/or require access to verifiable rewards or ground-truth answers over large candidate pools, which is costly and often infeasible in specialized domains. We study RLVR data selection in a setting where selection must be performed before any RL training and without labels or reward evaluation on the full pool. We propose SHIFT, a one-shot, training-free selector based solely on inference-time hidden-state dynamics. For each candidate instance, SHIFT runs a single deterministic reasoning rollout and computes a reasoning-induced representation shift (RIRS) as the start-to-end hidden-state delta. SHIFT uses the RIRS magnitude as a lightweight proxy for instance utility and enforces coverage via a quality-weighted farthest-first CoreSet procedure in an RIRS-augmented feature space, producing compact subsets that scale to large unlabeled pools. Across mathematical reasoning and medical QA benchmarks under ultra-low budgets, SHIFT consistently outperforms training-free diversity and difficulty/uncertainty baselines, improving both in-domain accuracy and transfer to harder evaluation settings. Ablations show that RIRS-based coverage and quality-weighting contribute complementary gains, and analyses indicate that RIRS is not explained by simple input/output length statistics. Code is available at github.com/JianghaoWu/SHIFT.

Large language models can improve their reasoning ability by training on examples with clearly checkable answers, such as math problems or medical exam questions. However, their improvement can depend strongly on which examples are chosen, and finding the best examples usually requires labels, answer checking, or extra training, all of which can be expensive. This paper proposes a simple way to choose useful training examples before any training is performed. For each candidate question, we let the model produce one answer and observe how much its internal representation changes during the reasoning process. Questions that cause stronger and more distinctive internal changes are treated as potentially more useful for later training. We then select a small but diverse set of such questions. Experiments on math reasoning and medical question answering show that this approach can find effective training examples using only a very small budget. It often improves model performance more than random selection or other training-free selection methods. This may help make reasoning-model adaptation cheaper and more practical in settings where labels or expert annotations are limited.