Suppression is not Deletion: Adversarial Probes Recover Unlearned Knowledge in Code LLMs

Code language models memorize library-API patterns from pretraining, and unlearning recipes are used to remove these patterns when libraries deprecate features. We do not yet know whether the recipes delete the underlying knowledge or only block its emission under direct prompting; the question has not been tested systematically for code LLMs. We introduce \textbf{CodeUnlearn-Bench}, a six-axis adversarial suite that runs from direct prompting up to per-layer probing, and apply it to four standard recipes on quantized 7B-class code LLMs: SFT, gradient ascent with retain regularization, ReLearn, and a Fisher-weighted task-vector variant. Linear probes recover deprecated-vs-current discrimination at every unlearning checkpoint, on par with the base model. The signal is recoverable behaviorally too: a small recovery fine-tune brings deprecated emission back, and the rebound replicates across two model families and two libraries. The recipes we tested suppress emission without erasing the underlying representation, so reporting only direct-prompt emission overstates how much knowledge has been removed.