How Good is Post-Hoc Watermarking With Language Model Rephrasing?

Generation-time text watermarking embeds statistical signals into text for traceability of AI-generated content. We explore post-hoc watermarking where an LLM rewrites existing text while applying generation-time watermarking, to protect copyrighted documents, or detect their use in training or RAG via watermark radioactivity. Unlike generation-time approaches which are constrained by how LLMs are served, this setting offers additional degrees of freedom for both generation and detection. We thus investigate how allocating compute (through larger rephrasing models, beam search, multi-candidate generation, or entropy filtering at detection) affects the quality-detectability trade-off. Among our findings, the simple Gumbel-max scheme surprisingly outperforms more recent alternatives under nucleus sampling, and achieves strong detectability and semantic fidelity on open-ended text such as books. Moreover, most methods benefit significantly from beam search, and we counterintuitively find that smaller models outperform larger ones. However, our solutions struggles when watermarking verifiable text such as code. This study reveals both the potential and limitations of post-hoc watermarking, laying groundwork for practical applications and future research.

Many people want ways to mark AI-written or AI-edited text so that its source can later be checked. This is useful for tracing copied material, protecting documents, or detecting whether protected text was used by another AI system. Our paper studies a simple way to do this for text that already exists: ask a language model to rewrite the text while quietly adding a hidden mark. We test how well this works across books, Wikipedia articles, multiple languages, and computer code. We find that the approach can work well for ordinary written text: the rewritten text usually keeps the same meaning, while the hidden mark remains detectable. Surprisingly, smaller language models can sometimes add stronger marks than larger ones, because they leave more room for variation in wording. We also find that some extra search during rewriting can improve the result. The method is much less reliable for code. Code must remain exactly correct, so there is less freedom to rewrite it without breaking it. Overall, our study maps when this kind of hidden marking is practical, when it fails, and which design choices work best.