De-AntiFake: Rethinking the Protective Perturbations Against Voice Cloning Attacks

The rapid advancement of speech generation models has heightened privacy and security concerns related to voice cloning (VC). Recent studies have investigated disrupting unauthorized voice cloning by introducing adversarial perturbations. However, determined attackers can mitigate these protective perturbations and successfully execute VC. In this study, we conduct the first systematic evaluation of these protective perturbations against VC under realistic threat models that include perturbation purification. Our findings reveal that while existing purification methods can neutralize a considerable portion of the protective perturbations, they still lead to distortions in the feature space of VC models, which degrades the performance of VC. From this perspective, we propose a novel two-stage purification method: (1) Purify the perturbed speech; (2) Refine it using phoneme guidance to align it with the clean speech distribution. Experimental results demonstrate that our method outperforms state-of-the-art purification methods in disrupting VC defenses. Our study reveals the limitations of adversarial perturbation-based VC defenses and underscores the urgent need for more robust solutions to mitigate the security and privacy risks posed by VC. The code and audio samples are available at https://de-antifake.github.io.

Voice cloning technology, which mimics a person's voice from their audio samples, is advancing rapidly, leading to privacy fears. To counter this, researchers have tried adding subtle, inaudible 'shields' to voice recordings to block unauthorized cloning.Our study investigates how well these protective shields hold up if someone tries to 'clean' the audio to remove them. We found that while existing cleaning methods can remove some of the shield, they often slightly distort the audio, which can impair the resulting voice clone. Building on this understanding, we created a novel two-stage cleaning method. This technique first purifies the recording by removing the shield, and then uses text content of speech as a guide to carefully restore the audio quality, making it more similar to the original voice without 'shields'.Our experiments show that this advanced cleaning process can effectively defeat current voice shields, enabling high-quality voice cloning. This highlights a critical vulnerability in current voice protection strategies and underscores the urgent need for more robust security measures to protect our voices.