WMVLM: Evaluating Diffusion Model Image Watermarking via Vision-Language Models

Digital watermarking is essential for securing generated images from diffusion models. Accurate watermark evaluation is critical for algorithm development, yet existing methods have significant limitations: they lack a unified framework for both residual and semantic watermarks, provide results without interpretability, neglect comprehensive security considerations, and often use inappropriate metrics for semantic watermarks. To address these gaps, we propose WMVLM, the first unified and interpretable evaluation framework for diffusion model image watermarking via vision-language models (VLMs). We redefine quality and security metrics for each watermark type: residual watermarks are evaluated by artifact strength and erasure resistance, while semantic watermarks are assessed through latent distribution shifts. Moreover, we introduce a three-stage training strategy to progressively enable the model to achieve classification, scoring, and interpretable text generation. Experiments show WMVLM outperforms state-of-the-art VLMs with strong generalization across datasets, diffusion models, and watermarking methods.

AI image generators can create realistic pictures at large scale, which makes it important to mark generated images so they can be traced and used responsibly. Researchers have proposed many watermarking methods, but judging whether a watermark is good is surprisingly difficult. Some watermarks leave small visible patterns in the image, while others are hidden in the generator’s internal sampling process, so existing evaluation tools often use incompatible scores and give little explanation. WMVLM addresses this by using a vision-language model to evaluate both kinds of watermarks in one framework. Instead of only reporting numbers, it identifies the watermark type, scores image quality and security, and explains what evidence in the image or generation process supports the score. For visible residual watermarks, it measures how much the mark affects the picture and whether it survives common edits such as compression, noise, and filtering. For hidden semantic watermarks, it checks whether the pattern of random values used during generation stays close to the behavior expected from an unwatermarked generator. The model is trained in stages to learn scoring, explanation, and generalization across watermark methods. Experiments show that WMVLM gives more accurate and more interpretable evaluations than strong existing vision-language models, helping watermark designers compare methods and build safer AI-image systems.