ACE-VC: Adaptive and Controllable Voice Conversion Using Explicitly Disentangled Self-Supervised Speech Representations
Authors:
- 4 June 2023
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 1-5
Abstract
In this work, we propose a zero-shot voice conversion method using speech representations trained with self-supervised learning. First, we develop a multi-task model to decompose a speech utterance into features such as linguistic content, speaker characteristics, and speaking style. To disentangle content and speaker representations, we propose a training strategy based on Siamese networks that encourages similarity between the content representations of the original and pitch-shifted audio. Next, we develop a synthesis model with pitch and duration predictors that can effectively reconstruct the speech signal from its decomposed representation. Our framework allows controllable and speaker-adaptive synthesis to perform zero-shot any-to-any voice conversion achieving state-of-the-art results on metrics evaluating speaker similarity, intelligibility, and naturalness. Using just 10 seconds of data for a target speaker, our framework can perform voice swapping and achieves a speaker verification EER of 5.5% for seen speakers and 8.4% for unseen speakers. 1Keywords
All Related Versions
This publication has 15 references indexed in Scilit:
- Quartznet: Deep Automatic Speech Recognition with 1D Time-Channel Separable Convolutions2020
- Voxceleb: Large-scale speaker verification in the wildComputer Speech & Language, 2020
- One-Shot Voice Conversion by Separating Speaker and Content Representations with Instance Normalization2019
- Average Modeling Approach to Voice Conversion with Non-Parallel Data2018
- SphereFace: Deep Hypersphere Embedding for Face Recognition2017
- An overview of voice conversion systemsSpeech Communication, 2017
- Phonetic posteriorgrams for many-to-one voice conversion without parallel data training2016
- Librispeech: An ASR corpus based on public domain audio books2015
- Connectionist temporal classification2006
- YIN, a fundamental frequency estimator for speech and musicThe Journal of the Acoustical Society of America, 2002