BPL: Generalizable Deepfake Detection via Bias-only Pair-aware Learning

The detection of synthetic images has traditionally been framed as a binary classification problem. However, we argue that this formulation overlooks a fundamental structural property of generative datasets: synthetic images are not independent samples, but are implicitly paired with real images sharing the same semantic source. Existing methods treat real and fake images as independent instances, failing to capture generation-induced relational discrepancies in real–fake pairs. Moreover, models tend to rapidly overfit to seen fake patterns, leading to poor generalization to unseen ones. To overcome these challenges, we propose a novel detection framework that explicitly mines real–fake pairs by constructing source-guided mappings or leveraging nearest-neighbor relationships in the CLIP embedding space. We then introduce pair-wise discrepancy learning that explicitly enlarges generation-induced deviations and discrepancy inversion to mitigate overfitting. Moreover, to preserve pretrained semantic representations while improving generalization, we adopt a bias-only fine-tuning scheme that restricts model capacity during adaptation. Extensive experiments show that our approach achieves superior generalization across unseen fake patterns.

As AI-generated images, especially realistic face forgeries known as deepfakes, become harder to detect, reliable detection tools are increasingly important for fighting misinformation and protecting trust in digital media. Many existing deepfake detectors learn by treating real and fake images as two separate groups. We suspected that this makes them memorize superficial patterns from the training data, causing them to fail when they encounter new kinds of deepfakes. Our paper introduces a framework for more generalizable deepfake detection. Instead of relying on complex architectural changes, we use paired data mining to find meaningful real–fake image pairs and use them to isolate the traces introduced by the generation process. This data-driven pairing helps the detector focus on forensic differences between aligned real and fake images, rather than memorizing dataset-specific or generator-specific patterns. At the same time, we preserve the general visual knowledge of a pre-trained vision model by updating only a very small set of bias parameters during training. This restricted adaptation allows the model to learn useful forgery cues while avoiding excessive changes to its original semantic understanding. Our experiments show that this paired approach makes deepfake detection more robust. In particular, it improves performance on new types of deepfakes that the model was not explicitly trained on, outperforming existing detection methods in several settings.