VSMask: Defending Against Voice Synthesis Attack via Real-Time Predictive Perturbation
Open Access
- 29 May 2023
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 239-250
Abstract
Deep learning based voice synthesis technology generates artificial human-like speeches, which has been used in deepfakes or identity theft attacks. Existing defense mechanisms inject subtle adversarial perturbations into the raw speech audios to mislead the voice synthesis models. However, optimizing the adversarial perturbation not only consumes substantial computation time, but it also requires the availability of entire speech. Therefore, they are not suitable for protecting live speech streams, such as voice messages or online meetings. In this paper, we propose VSMask, a real-time protection mechanism against voice synthesis attacks. Different from offline protection schemes, VSMask leverages a predictive neural network to forecast the most effective perturbation for the upcoming streaming speech. VSMask introduces a universal perturbation tailored for arbitrary speech input to shield a real-time speech in its entirety. To minimize the audio distortion within the protected speech, we implement a weight-based perturbation constraint to reduce the perceptibility of the added perturbation. We comprehensively evaluate VSMask protection performance under different scenarios. The experimental results indicate that VSMask can effectively defend against 3 popular voice synthesis models. None of the synthetic voice could deceive the speaker verification models or human ears with VSMask protection. In a physical world experiment, we demonstrate that VSMask successfully safeguards the real-time speech by injecting the perturbation over the air.Keywords
All Related Versions
Funding Information
- NSF (CNS-1950171)
This publication has 9 references indexed in Scilit:
- One-Class Learning Towards Synthetic Voice Spoofing DetectionIEEE Signal Processing Letters, 2021
- Voice-Indistinguishability -- Protecting Voiceprint with Differential Privacy under an Untrusted Server2020
- One-Shot Voice Conversion by Separating Speaker and Content Representations with Instance Normalization2019
- Voice Conversion from Unaligned Corpora Using Variational Autoencoding Wasserstein Generative Adversarial Networks2017
- An Algorithm for Intelligibility Prediction of Time–Frequency Weighted Noisy SpeechIEEE Transactions on Audio, Speech, and Language Processing, 2011
- Voice Conversion Using Partial Least Squares RegressionIEEE Transactions on Audio, Speech, and Language Processing, 2010
- Voice Conversion Based on Weighted Frequency WarpingIEEE Transactions on Audio, Speech, and Language Processing, 2009
- Noise cancelling for microphone arrays2002
- Echo cancellation and applicationsIEEE Communications Magazine, 1990