Differentially Private Parameter-Efficient Fine-tuning for Large ASR Models
Preprint
Abstract
Large ASR models can inadvertently leak sensitive information, which can be mitigated by formal privacy measures like differential privacy (DP). However, traditional DP training is computationally expensive, and can hurt model performance. Our study explores DP parameter-efficient fine-tuning as a way to mitigate privacy risks with smaller computation and performance costs for ASR models. Through extensive experimentation and progressive optimization, we achieve 4.6%/8.1% word error rate on LibriSpeech clean/other test-sets, setting a new performance benchmark while maintaining (10, 3.52e-6)-DP in fine-tuning a large ASR model with over 600M parameters.Keywords
All Related Versions
This publication has 10 references indexed in Scilit:
- Unintended Memorization in Large ASR Models, and How to Mitigate It2024
- Noise Masking Attacks and Defenses for Pretrained Speech Models2024
- Modular Domain Adaptation for Conformer-Based Streaming ASR2023
- How to DP-fy ML: A Practical Guide to Machine Learning with Differential PrivacyJournal of Artificial Intelligence Research, 2023
- Extracting Targeted Training Data from ASR Models, and How to Mitigate It2022
- BitFit: Simple Parameter-efficient Fine-tuning for Transformer-based Masked Language-models2022
- Conformer: Convolution-augmented Transformer for Speech Recognition2020
- Deep Learning with Differential Privacy2016
- Librispeech: An ASR corpus based on public domain audio books2015
- Private Empirical Risk Minimization: Efficient Algorithms and Tight Error Bounds2014