Memorization Removal as a Two-Player Game: The Adversarial Work Criterion as a Test for Foundation-Model Defenses

Recent work on memorization in diffusion models—Finding NeMo (Hintersdorf et al., 2024) and its follow-up Finding Dori (Kowalczuk et al., 2025)—presents a striking empirical pattern: a defense that suppresses memorized generation under the original training prompt can be defeated by adversarial embeddings, even though the defense “works” on every standard benchmark. We argue that this is not a contingent failure of NeMo or any specific localization method, but a structural consequence of evaluating memorization defenses against fixed prompts rather than against an adversary. We propose that the field adopt an Adversarial Work Criterion (AWC) that quantifies the computational work required to elicit memorized content from a frozen defended model, and that a defense be called effective only when this work scales exponentially in the resources of a bounded adversary. The AWC complements differential privacy (informationtheoretic, distribution-level) and membershipinference benchmarks (single-adversary, singlebudget) by providing a per-model, per-datum, computational lower bound. We give a toy energylandscape calculation showing that the AWC formally classifies NeMo-style local patches alongside generic gradient obfuscation—both scoring near zero—while reserving polynomial scores for defenses that genuinely flatten the memorization basin; this recovers the empirical finding of Finding Dori from the AWC formalism. The position is normative; we are honest about what is conjectural and what is provable