Breaking Manifold Continuity: Vector Quantized Modeling for Real-Centric Deepfake Detection

The increasingly realistic and diverse generative data has led some deepfake detection methods to shift towards learning robust real content, e.g., via reconstruction-based tasks. However, most existing approaches rely primarily on prevalent continuous modeling (e.g., GMMs, VAEs, Diffusion Models) to construct a continuous latent manifold of real data, with the aim of improving the generalization capability, while overlooking a critical issue, i.e., such continuity may facilitate the interpolation of forgery artifacts, consequently causing ambiguity in detection. To alleviate this problem, we integrate discrete modeling into the feature space of the CLIP vision encoder, striking a balance between continuous manifold modeling and discrete representation. By incorporating a learnable vector quantized codebook, the real latent manifold is discretized, imposing a more stringent information bottleneck that reduces the likelihood of embedding generative artifacts. In order to further enhance the generalization of discrete modeling, we propose an adaptive tangent space projection mechanism that yields a continuous relaxation of the discrete real distribution within a controllable range. With these components, our method constructs a real distribution that is both tightly constrained and broadly generalizable, enhancing robustness to unseen forgeries. Extensive experiments on diverse datasets demonstrate the effectiveness of our method.

With the rapid advances in generative models, deepfakes are becoming increasingly realistic and difficult to detect. Many existing detection methods focus on finding specific visual artifacts left by image generation systems. However, these methods often fail when new generation techniques appear. In this work, we take a different approach by focusing on learning the characteristics of real images instead of chasing every new type of fake. We observe that many existing systems represent real images too smoothly, which can unintentionally allow fake content to resemble real data. To address this issue, we design a new method that represents real images using a more structured and constrained feature space. This makes it harder for manipulated images to fit naturally into the learned representation. At the same time, our method still allows normal changes in real faces, such as different lighting conditions, poses, and expressions. Experiments on multiple benchmark datasets show that our approach is more robust to unseen deepfake generation methods and real-world image variations than previous techniques. We hope this work can help improve the reliability of deepfake detection systems and support efforts to reduce the spread of misleading AI-generated media.