Freeze and Learn: Continual Learning with Selective Freezing for Speech Deepfake Detection
Authors:
- 6 April 2025
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 1-5
Abstract
In speech deepfake detection, one of the critical aspects is developing detectors able to generalize on unseen data and distinguish fake signals across different datasets. Common approaches to this challenge involve incorporating diverse data into the training process or fine-tuning models on unseen datasets. However, these solutions can be computationally demanding and may lead to the loss of knowledge acquired from previously learned data. Continual learning techniques offer a potential solution to this problem, allowing the models to learn from unseen data without losing what they have already learned. Still, the optimal way to apply these algorithms for speech deepfake detection remains unclear, and we do not know which is the best way to apply these algorithms to the developed models. In this paper we address this aspect and investigate whether, when retraining a speech deepfake detector, it is more effective to apply continual learning across the entire model or to update only some of its layers while freezing others. Our findings, validated across multiple models, indicate that the most effective approach among the analyzed ones is to update only the weights of the initial layers, which are responsible for processing the input features of the detector.Keywords
All Related Versions
Funding Information
- Advanced Research Projects Agency
This publication has 20 references indexed in Scilit:
- Deepfake Media Forensics: State of the Art and Challenges Ahead2025
- All-for-One and One-for-All: Deep Learning-Based Feature Fusion for Synthetic Speech Detection2025
- A Robust Audio Deepfake Detection System via Multi-View Feature2024
- Class-Incremental Learning for Multi-Label Audio Classification2024
- Deepfake Generation and Detection: Case Study and ChallengesIEEE Access, 2023
- End-to-End anti-spoofing with RawNet22021
- FoR: A Dataset for Synthetic Speech Detection2019
- Speaker Recognition from Raw Waveform with SincNet2018
- Riemannian Walk for Incremental Learning: Understanding Forgetting and Intransigence2018
- Librispeech: An ASR corpus based on public domain audio books2015