Ghost Heads Across Training: When Greedy and Distributional Patching Disagree

A language model can produce a correct answer within 32 samples long before it does so reliably on the first try. Across arithmetic, GSM8K, and MATH500, pass@32 saturates during pretraining while pass@1 lags far behind. What mechanistic process closes this gap? We track circuit formation across OLMo-3-7B's full training pipeline (pretraining, mid-training, and RL-Zero) with both greedy and distributional activation patching, and introduce answer-token patching for multi-step reasoning. The two metrics agree on most components but dissociate at specific attention heads, which we call ghost heads: heads with high greedy recovery but near-zero distributional impact. Ghost heads are a recurring training instability: they peak whenever the training objective shifts and decay within each phase. Mid-training resolves the dissociation: patching effects concentrate onto a few attention heads where both metrics agree. RL-Zero, which improves pass@1 without touching pass@32, does not reintroduce ghost heads; two independent tests (noise sensitivity and verifier reranking) confirm it acts as a diffuse whole-model shift rather than reorganizing individual circuits. The phenomenon replicates across three model families.