Probability-Entropy Calibration: An Elastic Indicator for Adaptive Fine-tuning

Token-level reweighting is a simple yet effective mechanism for controlling supervised fine-tuning, but common indicators are largely one-dimensional: the ground-truth probability reflects downstream alignment, while token entropy reflects intrinsic uncertainty induced by the pre-training prior. Ignoring entropy can misidentify noisy or easily replaceable tokens as learning-critical, while ignoring probability fails to reflect target-specific alignment. RankTuner introduces a probability--entropy calibration signal, the Relative Rank Indicator, by comparing the rank of the ground-truth token with its expected rank under the predictiondiction distribution. The inverse indicator is used as a token-wise Relative Scale to reweight the fine-tuning objective, focusing updates on truly under-learned tokens without over-penalizing intrinsically uncertain positions. Experiments on multiple backbones show consistent improvements on mathematical reasoning benchmarks, transfer gains on out-of-distribution reasoning, and pre code generation performance over probability- or entropy-only reweighting baselines.

When a language model is trained to improve at math reasoning, it learns from many individual words and symbols in each solution. But some parts of a solution matter much more than others: a wrong number or equation step can break the answer, while a filler word or interchangeable phrase usually does not. Many current fine-tuning methods struggle to tell these cases apart because they look only at how confident the model is in the correct word, or only at how uncertain the model is overall. RankTuner gives the model a better sense of what deserves attention. For each token, it compares the rank of the correct answer token with the rank we would expect from the model’s own uncertainty. If the model makes a surprising mistake in a clear situation, RankTuner strengthens the learning signal; if the token is ambiguous or noisy, it weakens that signal. Experiments on multiple math reasoning benchmarks show that this helps models learn more reliably than standard fine-tuning and previous token-weighting methods.