LORD-GoF: A Robust Online Detection Approach for LLM Watermarks in Sparse and Mixed Streams

Watermarking is crucial for identifying AI-generated text, however, existing detection methods often focus on offline settings and fail to control the online False Discovery Rate (oFDR) when applied to real-world streams where machine-generated content is sparse and mixed with human writing. To address this issue, in this paper, we propose LORD-GoF, a novel online detection framework that combines a Goodness-of-Fit (GoF) statistic with the Levels based On Recent Discovery (LORD) procedure. We prove that LORD-GoF approach can rigorously control the oFDR below a user-specified level by dynamically adjusting detection thresholds. Extensive experiments on watermarked text from Qwen-2.5-3B, Sheared-LLaMA-2.7B, and OPT-1.3B using both the Gumbel-Max and Inverse Transform watermarking schemes show that our method maintains statistical power comparable to offline benchmarks while successfully controlling the oFDR under complex, mixed streaming scenarios.

Large language models can now write text that looks very similar to human writing. One way to identify AI-generated text is to add a hidden watermark when the text is produced. However, in real use, detection is not done on a clean and balanced test set. A platform may see a long stream of documents, most of which are written by humans, while only a small number contain watermarks. Some AI-generated texts may also be edited before detection. In this setting, using the same fixed detection rule for every document can create too many false alarms. Our paper proposes LORD-GoF, a method that adjusts its detection threshold over time. It is designed to find watermarked documents while keeping mistaken flags under control. Experiments on several language models and watermarking methods show that LORD-GoF works well in sparse and mixed text streams, reducing false alarms while still keeping strong detection ability.