Attacks on Machine-Text Detectors Retain Stylistic Fingerprints

Despite considerable progress in the development of machine-text detectors, the ease with which machine-text can be manipulated to evade detection has led to suggestions that the problem is inherently intractable. In this work, we investigate the limits of such evasion strategies. We demonstrate that while current attacks, ranging from prompt engineering to detector-guided optimization can effectively degrade performance of standard detectors, they fail to erase the underlying stylistic "fingerprints" of machine text. We show that few-shot detectors that utilize the stylistic feature space are robust to these evasion attempts, reliably detecting samples even from models explicitly tuned to prevent detection. This raises the question: does style represent a universal defense against machine-detection attacks? We demonstrate that the answer is "no" by introducing a novel paraphrasing approach that simultaneously optimizes for undetectability and adherence to specific human styles. We show that unlike prior methods, this attack effectively evades all considered detectors, including those that utilize writing style. However, we find that this evasion is not absolute: as the number of documents available for analysis grows, the human and machine distributions become distinguishable again. Overall, our findings suggest that reliable machine-text detection requires moving beyond single-document analysis to multi-document analysis.

Can AI-written text always be detected, or can it be rewritten to look completely human? We explored this question by testing a wide range of methods designed to hide the fact that text was generated by AI. We found that most existing tricks can fool standard detectors, but they still leave behind hidden writing patterns, similar to a fingerprint. Systems that focus on writing style can often still recognize the text as machine-generated, even when the AI was trained specifically to avoid detection. But we also developed a stronger attack: a rewriting system that changes AI-generated text to closely mimic human writing styles. This new method was able to evade every detector we tested, including style-based ones. Even so, the deception does not fully hold up when many documents from the same source are examined together. Our findings suggest that future AI-text detection systems should focus on long-term writing patterns across multiple documents, rather than relying on a single piece of text.