LLM Watermark Evasion via Bias Inversion

Watermarking offers a promising solution for detecting LLM-generated content, yet its robustness under realistic query-free (black-box) evasion remains an open challenge. Existing query-free attacks often achieve limited success or severely distort semantic meaning. We bridge this gap by theoretically analyzing rewriting-based evasion, demonstrating that reducing the average conditional probability of sampling green tokens by a small margin causes the detection probability to decay exponentially. Guided by this insight, we propose the \emph{Bias-Inversion Rewriting Attack} (BIRA), a practical query-free method that applies a negative logit bias to a proxy suppression set identified via token surprisal. Empirically, BIRA achieves state-of-the-art evasion rates ($>99\%$) across diverse watermarking schemes while preserving semantic fidelity substantially better than prior baselines. Our findings reveal a fundamental vulnerability in current watermarking methods and highlight the need for rigorous stress tests. Our code is available at \href{https://github.com/ml-postech/LLM-Watermark-Evasion-via-Bias-Inversion}{here}.

How reliable are watermarks for detecting text written by large language models? Watermarking is a promising way to identify AI-generated content and prevent misuse, such as misinformation generation, by subtly biasing which tokens a model is likely to choose. However, in real-world settings, watermarked text can be easily rewritten. This raises an important question: are current watermarking methods truly reliable under such attacks? Our paper shows that many current watermarking methods are more fragile than expected. We first provide a theoretical explanation: even a small reduction in the probability of choosing watermark-favored tokens can cause detection success to drop rapidly. Based on this insight, we introduce Bias-Inversion Rewriting Attack (BIRA), a simple black-box rewriting method that discourages certain high-surprisal tokens during paraphrasing. This allows the rewritten text to evade watermark detection while largely preserving its original meaning. In experiments, BIRA removes watermarks across a range of recent watermarking schemes much more effectively than previous attacks, while maintaining better semantic fidelity. These findings suggest that current watermarking methods need stronger robustness guarantees and more realistic stress tests before they can be relied on in practice.