Improved DeepFake Detection Using Whisper Features
Preprint
Abstract
With a recent influx of voice generation methods, the threat introduced by audio DeepFake (DF) is ever-increasing. Several different detection methods have been presented as a countermeasure. Many methods are based on so-called front-ends, which, by transforming the raw audio, emphasize features crucial for assessing the genuineness of the audio sample. Our contribution contains investigating the influence of the state-of-the-art Whisper automatic speech recognition model as a DF detection front-end. We compare various combinations of Whisper and well-established front-ends by training 3 detection models (LCNN, SpecRNet, and MesoNet) on a widely used ASVspoof 2021 DF dataset and later evaluating them on the DF In-The-Wild dataset. We show that using Whisper-based features improves the detection for each model and outperforms recent results on the In-The-Wild dataset by reducing Equal Error Rate by 21%.Keywords
All Related Versions
This publication has 13 references indexed in Scilit:
- ASVspoof 2021: Towards Spoofed and Deepfake Speech Detection in the WildIEEE/ACM Transactions on Audio, Speech, and Language Processing, 2023
- The Vicomtech Spoofing-Aware Biometric System for the SASV Challenge2022
- Fully Automated End-to-End Fake Audio Detection2022
- Deepfakes generation and detection: state-of-the-art, open challenges, countermeasures, and way forwardApplied Intelligence, 2022
- ASVspoof 2021: accelerating progress in spoofed and deepfake speech detection2021
- End-to-End anti-spoofing with RawNet22021
- Tandem Assessment of Spoofing Countermeasures and Automatic Speaker Verification: FundamentalsIEEE/ACM Transactions on Audio, Speech, and Language Processing, 2020
- A Light CNN for Deep Face Representation With Noisy LabelsIEEE Transactions on Information Forensics and Security, 2018
- Constant Q cepstral coefficients: A spoofing countermeasure for automatic speaker verificationComputer Speech & Language, 2017
- Design, analysis and experimental evaluation of block based transformation in MFCC computation for speaker recognitionSpeech Communication, 2012