Beyond the Screen: Evaluating Deepfake Detectors under Moiré Pattern Effects
Authors:
- 17 June 2024
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 4429-4439
Abstract
The detection of deepfakes is crucial for mitigating the societal impact of falsified video content. Despite the development of various algorithms for this purpose, challenges arise for detectors in real-world scenarios, especially when users capture deepfake content from screens and upload it online or when detectors operate on external devices like smartphones, requiring the capture of potential deepfakes through the camera for evaluation. A significant challenge in these scenarios is the presence of Moiré patterns, which degrade image quality and complicate conventional classification methods, notably deep neural networks (DNNs). However, the impact of Moiré patterns on the effectiveness of deepfake detection systems has not been adequately explored. This study aims to investigate how capturing deepfake videos via digital screen cameras affects the accuracy of detection mechanisms. We introduced the Moiré patterns by capturing the display of a monitor using a smartphone camera and conducted empirical evaluations using four widely recognized datasets: CelebDF, DFD, DFDC, and FF++. We compare the performance of twelve SOTA detectors on deepfake videos captured under the influence of Moiré patterns. Our findings reveal a performance decrease of up to 33.1 and 31.3 percentage points for image- and video-based detectors. Therefore, highlighting the challenges posed by Moiré patterns and other naturally induced artifacts is critical for improving the effectiveness of real-world deepfake detection efforts. To facilitate further research, we will release the Moiré pattern impact version of CelebDF, DFD, DFDC, and FF++ datasets with this paper. Our code is available here: https://github.com/Razaib-Tariq/deepmoireKeywords
Funding Information
- Sungkyunkwan University
- Sungkyunkwan University
- Research and Development
This publication has 43 references indexed in Scilit:
- A Survey on the Detection and Impacts of Deepfakes in Visual, Audio, and Textual FormatsIEEE Access, 2023
- Combining EfficientNet and Vision Transformers for Video Deepfake Detection2022
- Detecting handcrafted facial image manipulations and GAN-generated facial images using Shallow-FakeFaceNetApplied Soft Computing, 2021
- TAR: Generalized Forensic Framework to Detect Deepfakes Using Weakly Supervised Learning2021
- Multi-attentional Deepfake Detection2021
- Improving the Efficiency and Robustness of Deepfakes Detection through Precise Geometric Features2021
- Deepfake Video Detection Using Audio-Visual Consistency2021
- Deepfakes and beyond: A Survey of face manipulation and fake detectionInformation Fusion, 2020
- Celeb-DF: A Large-Scale Challenging Dataset for DeepFake Forensics2020
- Deepfake news: AI-enabled disinformation as a multi-level public policy challengeJournal of Cyber Policy, 2020