Improving Deepfake Detection Generalization by Invariant Risk Minimization
- 19 January 2024
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Multimedia
- Vol. 26 (15209210) , 6785-6798
Abstract
The abuse of deepfake techniques has raised serious concerns about social security and ethical problems, which motivates the development of deepfake detection. However, without fully addressing the domain gap issue, existing deepfake detection methods still show weak generalization ability among datasets belonging to different domains with domain-specific characteristics like identities and generation methods, limiting their practical applications. In this paper, we propose the Invariant Domain-oriented Deepfake Detection method (ID$_{3}$ ) , which improves the generalization of deepfake detection on multiple domains through invariant risk minimization, a novel learning paradigm that addresses the domain gap problem by jointly training a purified invariant predictor and learning an aligned invariant representation. To train a purified invariant predictor, we design the Domain Refinement Data Augmentation strategy with self-face-swapping and region-erasing approaches, which suppresses domain-specific features and encourages the models to focus on critical domain-invariant characteristics. To learn an aligned invariant representation, we propose the Domain Calibration Batch Normalization approach with multiple BN branches, which normalizes input features from different domains into aligned representations during both training and testing. Extensive experiments on multiple datasets demonstrate that our framework can boost the deepfake detection generalization ability and outperform other baselines by large margins. Our codes can be found here
Keywords
This publication has 52 references indexed in Scilit:
- Towards Defending Multiple $\ell _p$-Norm Bounded Adversarial Perturbations via Gated Batch NormalizationInternational Journal of Computer Vision, 2023
- Face reenactment via generative landmark guidanceImage and Vision Computing, 2022
- Understanding the Security of Deepfake Detection2022
- Detecting deepfake videos based on spatiotemporal attention and convolutional LSTMInformation Sciences, 2022
- Deepfakes and beyond: A Survey of face manipulation and fake detectionInformation Fusion, 2020
- Deepfake Video Detection Using Recurrent Neural Networks2018
- Adaptive Batch Normalization for practical domain adaptationPattern Recognition, 2018
- StarGAN: Unified Generative Adversarial Networks for Multi-domain Image-to-Image Translation2018
- Xception: Deep Learning with Depthwise Separable Convolutions2017
- Face2Face: Real-Time Face Capture and Reenactment of RGB Videos2016