The Codecfake Dataset and Countermeasures for the Universally Detection of Deepfake Audio
Top Cited Papers
- 7 January 2025
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Audio, Speech and Language Processing
- Vol. 33, 386-400
Abstract
With the proliferation of Audio Language Model (ALM) based deepfake audio, there is an urgent need for generalized detection methods. ALM-based deepfake audio currently exhibits widespread, high deception, and type versatility, posing a significant challenge to current audio deepfake detection (ADD) models trained solely on vocoded data. To effectively detect ALM-based deepfake audio, we focus on the mechanism of the ALM-based audio generation method, the conversion from neural codec to waveform. We initially constructed the Codecfake dataset, an open-source, large-scale collection comprising over 1 million audio samples in both English and Chinese, focus on ALM-based audio detection. As countermeasure, to achieve universal detection of deepfake audio and tackle domain ascent bias issue of original sharpness aware minimization (SAM), we propose the CSAM strategy to learn a domain balanced and generalized minima. In our experiments, we first demonstrate that ADD model training with the Codecfake dataset can effectively detects ALM-based audio. Furthermore, our proposed generalization countermeasure yields the lowest average equal error rate (EER) of 0.616% across all test conditions compared to baseline models. The dataset1 and associated code2 are available online.Keywords
All Related Versions
Funding Information
- National Natural Science Foundation of China (62101553, 62306316, U21B20210, 62201571)
This publication has 47 references indexed in Scilit:
- CFAD: A Chinese dataset for fake audio detectionSpeech Communication, 2024
- TIMIT-TTS: A Text-to-Speech Dataset for Multimodal Synthetic Media DetectionIEEE Access, 2023
- w2v-BERT: Combining Contrastive Learning and Masked Language Modeling for Self-Supervised Speech Pre-Training2021
- Upsampling Artifacts in Neural Audio Synthesis2021
- Parallel Wavegan: A Fast Waveform Generation Model Based on Generative Adversarial Networks with Multi-Resolution Spectrogram2020
- Waveglow: A Flow-based Generative Network for Speech Synthesis2019
- Natural TTS Synthesis by Conditioning Wavenet on MEL Spectrogram Predictions2018
- Librispeech: An ASR corpus based on public domain audio books2015
- Deep neural networks for small footprint text-dependent speaker verification2014
- Long Short-Term Memory2012