Detecting Deepfake Videos using Spatiotemporal Trident Network
Open Access
- 12 September 2024
- journal article
- research article
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Multimedia Computing, Communications, and Applications
- Vol. 20 (11) , 1-20
Abstract
The widespread dissemination of Deepfake in social networks has posed serious security risks, thus necessitating the development of an effective Deepfake detection technique. Currently, video-based detectors have not been explored as extensively as image-based detectors. Most existing video-based methods only consider temporal features without combining spatial features, and do not mine deeper-level subtle forgeries, resulting in limited detection performance. In this paper, a novel spatiotemporal trident network (STN) is proposed to detect both spatial and temporal inconsistencies of Deepfake videos. Since there is a large amount of redundant information in Deepfake video frames, we introduce convolutional block attention module (CBAM) on the basis of the I3D network and optimize the structure to make the network better focus on the meaningful information of the input video. Aiming at the defects in the deeper-level subtle forgeries, we designed three feature extraction modules (FEMs) of RGB, optical flow, and noise to further extract deeper video frame information. Extensive experiments on several well-known datasets demonstrate that our method has promising performance, surpassing several state-of-the-art Deepfake video detection methods.Keywords
Funding Information
- National Key Research and Development Plan (No. 2019QY1406)
- National Natural Science Foundation of China (61972106, 61902082)
- Major Key Project of PCL (PCL2022A03-3, PCL2021A02)
- Dongguan Innovative Research Team Program (2018607201008)
- Guangzhou Key Research and Development Plan (202206030001)
- Guangdong Higher Education Innovation Group (2020KCXTD007)
- Guangzhou Higher Education Innovation Group (202032854)
- Key Laboratory of the Education Department of Guangdong Province (2019KSYS009)
- Scientific and Technological Planning Projects of Guangdong Province (2021A0505030074)
- Key Construction Discipline Scientific Research Capacity Improvement Project of Guangdong Province (2021ZDJS025)
- Postgraduate Education Innovation Plan Project of Guangdong Province (2020SFKC054)
- Guangzhou University Graduate Innovation Research Funding Program (2022GDJC-D19)
This publication has 39 references indexed in Scilit:
- Two-Stream Neural Networks for Tampered Face Detection2017
- Microexpression Identification and Categorization Using a Facial Dynamics MapIEEE Transactions on Affective Computing, 2017
- Joint Face Detection and Alignment Using Multitask Cascaded Convolutional NetworksIEEE Signal Processing Letters, 2016
- Face segmentation in thumbnail images by data-adaptive convolutional segmentation networks2016
- Demo of Face2Face2016
- Learning Spatiotemporal Features with 3D Convolutional Networks2015
- A Main Directional Mean Optical Flow Feature for Spontaneous Micro-Expression RecognitionIEEE Transactions on Affective Computing, 2015
- Rich Models for Steganalysis of Digital ImagesIEEE Transactions on Information Forensics and Security, 2012
- Occlusion-Aware Optical Flow EstimationIEEE Transactions on Image Processing, 2008
- A Duality Based Approach for Realtime TV-L 1 Optical Flow2007