One-Shot Voice Conversion Algorithm Based on Representations Separation
Open Access
- 27 October 2020
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 8, 196578-196586
Abstract
Voice Conversion (VC) is a method of converting the source speaker’s speech into the target speaker’s speech without changing the source speaker’s speech content. The current VC methods have the following problems: (1) they are only applicable to a limited number of speakers, not to any speakers, as a result, the application scenarios are greatly restricted; (2) the representation (feature) separation(RS) effect of the current mainstream technology is not ideal on the source speaker speech and the target speaker speech; and (3) the voice conversion quality of most models is unsatisfactory, and hence needs to be improved. Therefore, in this paper, we constructed a one-shot VC model of Representation Separation, called RS-VC model, implemented by the encoder-decoder structure. The encoder is composed of a content encoder and a speaker encoder. The content encoder separates the content information of the source speaker voice and generates a content representation. The speaker encoder separates the target speaker information of the target speaker voice and generates a speaker representation. The decoder synthesizes the content representation and the speaker representation to generate the converted voice. In this paper, we obtained the optimized speaker verification model SVIGEN2E (Speaker Verification with Instance Normalization using Generalized End-to-End loss) by improving the speaker verification (SV) model. The model SVIGEN2E is used as the speaker encoder. This speaker encoder needs to be trained in advance prior to RS-VC model training, and the pre-trained model of SVINGE2E directly extracts speaker representation of the target speaker’s voice, and is used for training and testing RS-VC model. A progressive training method is proposed then for training RS-VC model. Experiments show that the progressive training method can effectively improve the quality of the converted voice. Compared with the basic speaker verification model, both SVINGE2E and RS-VC deliver the impressive improvements in EER (Equal Error Rate).Keywords
Funding Information
- Department of Education of Guangdong Province through Special Innovation Program (2015KTSCX183)
- South China University of Technology through “Development Fund” (x2js-F8150310)
This publication has 12 references indexed in Scilit:
- Multi-target Voice Conversion without Parallel Data by Adversarially Learning Disentangled Audio Representations2018
- WaveNet Vocoder with Limited Training Data for Voice Conversion2018
- Generalized End-to-End Loss for Speaker Verification2018
- Voice Conversion from Unaligned Corpora Using Variational Autoencoding Wasserstein Generative Adversarial Networks2017
- VoxCeleb: A Large-Scale Speaker Identification Dataset2017
- Librispeech: An ASR corpus based on public domain audio books2015
- GMM-based voice conversion with explicit modelling on feature transform2010
- Voice conversion algorithm based on Gaussian mixture model with dynamic frequency warping of STRAIGHT spectrum2001
- Statistical analysis of bilingual speaker’s speech for cross-language voice conversionThe Journal of the Acoustical Society of America, 1991
- Speaker adaptation and voice conversion by codebook mapping1991