Abstract
Deep learning has been successfully applied in various fields, and its impact on deepfake detection is no exception. Deepfakes are fake, yet realistic synthetic content that can be used deceitfully for political impersonation, phishing, slander, or the spread of misinformation. Despite extensive research on unimodal deepfake detection, the identification of complex deepfakes through joint analysis of audio and visual streams remains relatively unexplored. To fill this gap, this survey first provides an overview of audiovisual deepfake generation techniques, applications, and their consequences, and then provides a comprehensive review of state-of-the-art methods that combine audio and visual modalities to increase detection accuracy, summarizing and critically analyzing their strengths and limitations. Furthermore, we discuss existing open source datasets for a deeper understanding, which can contribute to the research community and provide necessary information for beginners who want to analyze deep learning-based audiovisual methods for video forensics. By bridging the gap between unimodal and multimodal approaches, this paper aims to improve the effectiveness of deepfake detection strategies and guide future research on cybersecurity and media integrity.
Funding Information
  • National Council of Science and Technology, Taiwan (NSTC 111-2221-E-001-002)
  • National Council of Science and Technology, Taiwan (NSTC 112-2221-E-001-009-MY3)
  • National Council of Science and Technology, Taiwan (NSTC 113-2221-E004-001-MY3)
  • National Council of Science and Technology, Taiwan (NSTC 113-2221-E-004-006-MY2)
  • National Council of Science and Technology, Taiwan (NSTC 114-2221-E-001-006-MY3)