Linear Ensembles Wash Away Watermarks: On the Fragility of Distributional Perturbations in LLMs
Abstract
Watermarking embeds statistical signatures in AI-generated text for detection and attribution. We reveal a fundamental vulnerability: when users access multiple models (today's reality), watermarks trivially fail. Watermarks perturb output distributions away from the original, and in competitive markets, these perturbations are typically independent across providers. We theoretically prove that averaging output probability distributions recovers the unwatermarked distribution with up to a second-order error term. Empirically, simply averaging 3-5 models cancels out these perturbations. We introduce WASH (Watermark Attenuation via Statistical Hybridisation), which solves practical challenges in ensemble generation: vocabulary misalignment and tokenisation differences across heterogeneous models. Experiments across six watermarking schemes and three LLMs show that averaging across 3 models suppresses detection z-scores from 5-300 to below 2 (below the detection threshold of 4) and reduces TPR@5%FPR to below 50%, while improving quality by 27.5% and running 6× faster than the best baseline on the long sequence generation. Our results suggest that robust AI-text detection via watermarking requires either accepting this fundamental vulnerability or unprecedented coordination among model providers.
Lay Summary
AI-generated writing is becoming difficult to distinguish from human writing, and one proposed safeguard is to hide a statistical “watermark” in the text. The watermark is not visible to readers, but a detector can look for tiny patterns in which words the model tends to choose. Our work asks what happens when a user can draw from several AI systems at the same time, as is common today. We find that watermarks from different providers often push word choices in different directions. When their predictions are averaged, these pushes can cancel, much like mixing several faint signals until none remains clear. We prove this effect mathematically and introduce WASH, a practical way to combine models even when they use different vocabularies and split words differently. In experiments with six watermarking methods and three language models, only a few models were enough to make watermark detectors unreliable while keeping the output fluent and useful. This suggests that watermarking cannot be treated as a single-provider fix for identifying AI text. For watermarking to support goals such as academic integrity, content authenticity, and intellectual property protection, model providers may need shared standards or coordinated signals rather than separate private watermarks.