SAFE: Sequential Attentive Face Embedding with Contrastive Learning for Deepfake Video Detection
Open Access
- 21 October 2023
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 3993-3997
Abstract
The emergence of hyper-realistic deepfake videos has raised significant concerns regarding their potential misuse. However, prior research on deepfake detection has primarily focused on image-based approaches, with little emphasis on video. With the advancement of generation techniques enabling intricate and dynamic manipulation of entire faces as well as specific facial components in a video sequence, capturing dynamic changes in both global and local facial features becomes crucial in detecting deepfake videos. This paper proposes a novel sequential attentive face embedding, SAFE, that can capture facial dynamics in a deepfake video. The proposed SAFE can effectively integrate global and local dynamics of facial features revealed in a video sequence using contrastive learning. Through a comprehensive comparison with the state-of-the-art methods on the DFDC (Deepfake Detection Challenge) dataset and the FaceForensic++ benchmark, we show that our model achieves the highest accuracy in detecting deepfake videos on both datasets.Keywords
Funding Information
- National Research Foundation of Korea (2021R1A4A3022102, 2023R1A2C2007625)
- Institute for Information and Communications Technology Promotion (RS-2023-00230337)
This publication has 31 references indexed in Scilit:
- Deepfake Video Detection through Optical Flow Based CNN2019
- FaceForensics++: Learning to Detect Manipulated Facial Images2019
- Deferred neural renderingACM Transactions on Graphics, 2019
- MesoNet: a Compact Facial Video Forgery Detection Network2018
- Deepfake Video Detection Using Recurrent Neural Networks2018
- Mesoscopic Facial Geometry Inference Using Deep Neural Networks2018
- Fast Face-Swap Using Convolutional Neural Networks2017
- Xception: Deep Learning with Depthwise Separable Convolutions2017
- Demo of Face2Face2016
- Rethinking the Inception Architecture for Computer Vision2016