Out of Context Obfuscation: What Facts Matter for Probe Evasion?

We ask whether language models can evade white-box monitors solely by leveraging factual knowledge of how those monitors work and where they fail. To measure this, we start with a malign initialization, where the LM is trained to hide adverse drug findings in a clinical reporting task. We then teach the model various facts about a fictitious white-box monitoring system within this environment, and evaluate whether teaching these facts results in a drop in a lie-detection probe's recall. Across Qwen-3-14B, -32B, and Llama-3.3-70B-Instruct, we find that language models successfully evade probes only when they are taught a concrete evasion strategy and given a strong incentive to deploy it — removing either collapses the effect. Because this type of evasion requires no optimization and only knowledge of how the probe fails, a potentially scheming model that already has the propensity to do so could plausibly acquire the same capability through pretraining alone, supporting calls to filter such content from pretraining corpora.