Understanding Audiovisual Deepfake Detection: Techniques, Challenges, Human Factors, and Perceptual Insights
- 9 April 2026
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Computational Intelligence Magazine
- Vol. 21 (2) , 38-54
Abstract
Deep learning has been successfully applied in various fields, and its impact on deepfake detection is no exception. Deepfakes are fake, yet realistic synthetic content that can be used deceitfully for political impersonation, phishing, slander, or the spread of misinformation. Despite extensive research on unimodal deepfake detection, the identification of complex deepfakes through joint analysis of audio and visual streams remains relatively unexplored. To fill this gap, this survey first provides an overview of audiovisual deepfake generation techniques, applications, and their consequences, and then provides a comprehensive review of state-of-the-art methods that combine audio and visual modalities to increase detection accuracy, summarizing and critically analyzing their strengths and limitations. Furthermore, we discuss existing open source datasets for a deeper understanding, which can contribute to the research community and provide necessary information for beginners who want to analyze deep learning-based audiovisual methods for video forensics. By bridging the gap between unimodal and multimodal approaches, this paper aims to improve the effectiveness of deepfake detection strategies and guide future research on cybersecurity and media integrity.Keywords
All Related Versions
Funding Information
- National Council of Science and Technology, Taiwan (NSTC 111-2221-E-001-002)
- National Council of Science and Technology, Taiwan (NSTC 112-2221-E-001-009-MY3)
- National Council of Science and Technology, Taiwan (NSTC 113-2221-E004-001-MY3)
- National Council of Science and Technology, Taiwan (NSTC 113-2221-E-004-006-MY2)
- National Council of Science and Technology, Taiwan (NSTC 114-2221-E-001-006-MY3)
This publication has 118 references indexed in Scilit:
- Pixels of Faith: Exploiting Visual Saliency to Detect Religious Image Manipulation2025
- PolyGlotFake: A Novel Multilingual and Multimodal DeepFake Dataset2024
- Deepfakes generation and detection: state-of-the-art, open challenges, countermeasures, and way forwardApplied Intelligence, 2022
- Deepfake Video Detection Using Audio-Visual Consistency2021
- Two-Branch Recurrent Network for Isolating Deepfakes in Videos2020
- The Malicious Use of AI-Based Deepfake Technology as the New Threat to Psychological Security and Political Stability2020
- You Said That?: Synthesising Talking Faces from AudioInternational Journal of Computer Vision, 2019
- ReenactGAN: Learning to Reenact Faces via Boundary Transfer2018
- Assessing the importance of audio/video synchronization for simultaneous translation of video sequencesMultimedia Systems, 2012
- Joint Face Alignment with Non-parametric Shape Models2012