Subliminal Transfer of Positional Biases in Language Models

Subliminal learning describes the transmission of behavioral preferences from teacher to student language models through training data containing no surface reference to the preference. Prior work has characterized this transfer for semantic preferences but leaves open whether it extends to structural biases like answer-letter preference in multiple-choice questions. We test this extension across 8 instruction-tuned LMs, running the full behavioral, mechanistic, and steering pipeline on each. Behaviorally, subliminal letter-bias transfer is observed in 7 of 8 models, with targetletter shifts ranging from below 0.2 pp to roughly 10 pp. We then conduct an in-depth mechanistic analysis on Qwen2.5-7B: masking divergence tokens reduces the bias shift 4× to within noise, token-entanglement geometry predicts which letters transfer most strongly, and the bias localizes to a residual-stream direction at late layers that admits causal activation steering. Replicating the steering analysis across the remaining models reveals five distinct mechanistic regimes, including a dissociation in Phi-3-medium where a clean, controllable bias representation does not manifest as behavioral letter selection. This dissociation shows that parameter-pull subliminal transfer can succeed at the representational level without affecting behavior, meaning behavioral evaluation alone cannot certify that a distilled model is free of inherited biases. Together, these results widen the safety-relevant scope of subliminal learning and underscore that the full extent of what distilled models inherit from their teachers cannot, at present, be assessed by behavioral auditing alone.