Stability-Aware Feature Design for Robust Watermark Detection in Machine-Generated Text

The widespread adoption of large language models (LLMs) has intensified the demand for principled methods to distinguish human- from machine-generated text. Watermarking provides a promising avenue, yet existing detectors exhibit sharp performance deterioration under multiple paraphrasing and when applied to shorter texts. We introduce Pattern Stability Score (PSS), a novel detection framework that leverages local statistical features and stability dynamics across paraphrased variants. Specifically, the proposed method combines global and local z-score features with higher-order statistics of run-length patterns, enriched by autocorrelation signals and stability scores computed over paraphrase depth. Numerical evaluations are performed on three benchmark datasets (PG-19, CNN/DailyMail, and WikiText) using multiple LLMs (Llama-3-8B, Qwen2-7B) and paraphrasers (Mistral-7B, Qwen2-7B, Gemma-7B), systematically stress-testing robustness under up to eight rounds of paraphrasing. Compared to prior z-score thresholding baselines and some state-of-the-art deep learning methods, our approach improves detection AUC (area under the receiver operating characteristic curve) by over 10-15 percentage points across different token lengths. Additionally, extensive cross-domain experiments demonstrate that a single universal classifier generalizes across different LLMs, paraphrasers, and text domains without retraining, maintaining above 83.7\% AUC even when all components differ from training.

As large language models (LLMs) become widely used, distinguishing human-written from AI-generated text is increasingly important. While watermarking offers a promising solution by embedding hidden signals in generated text, existing detection methods often fail when the text is paraphrased or very short. This paper proposes a new approach, called Pattern Stability Score (PSS), which detects AI-generated text by analyzing subtle statistical patterns and how consistently these patterns persist across multiple paraphrased versions. Extensive experiments across diverse datasets, models, and paraphrasing systems show that PSS substantially improves detection accuracy (by over 10-15 percentage points) and remains robust even under repeated paraphrasing. Notably, the method generalizes well across different models and domains without retraining, achieving strong performance (above 87.8% AUC), indicating its potential as a reliable and broadly applicable tool for identifying AI-generated text.