Why Inoculation Prompting Fails on Sycophancy

Sycophancy, the tendency of instruction-tuned language models to affirm users’ false claims, is a persistent failure mode in model alignment. Inoculation prompting (IP) fine-tunes models on prompts that explicitly elicit an unwanted behavior, aiming to suppress that behavior when the eliciting instruction is absent at test time. Prior work reports promising results for reward hacking and backdoors, where suppression is the core objective. For sycophancy, however, suppression is insufficient: models must reject the false premise and provide the correct answer. On a reconstructed greatest-common-divisor benchmark designed to prevent shortcut learning, we find that IP rarely produces genuine correction and mainly produces \textbf{behavioral substitution}: the model refuses the task, gives no usable answer, preserves the user's false answer or reasoning despite appearing to disagree, or fails to make a clear commitment. Specifically, across 16 IP prompts per placement, 2 prompt placements, and 4 seeds, behavioral substitution appears whenever sycophantic agreement falls at least 10 percentage points (pp) below the non-inoculated baseline. We find that placing IP prompts after the user's message reduces sycophantic agreement the most, by 20.9 pp on average and up to 55.3 pp for individual prompts relative to the non-inoculated baseline, while genuine-correction response rates remain below 0.6\% in every condition. We further find that base-model elicitation scores weakly predict downstream inoculation effects of specific IP prompts and providing only a coarse signal about IP prompt placement. These results suggest that sycophancy is too coarse a target for direct inoculation and effective interventions need to target component behaviors separately.