Voice Conversion from Non-parallel Corpora Using Variational Auto-encoder
Preprint
Abstract
We propose a flexible framework for spectral conversion (SC) that facilitates training with unaligned corpora. Many SC frameworks require parallel corpora, phonetic alignments, or explicit frame-wise correspondence for learning conversion functions or for synthesizing a target spectrum with the aid of alignments. However, these requirements gravely limit the scope of practical applications of SC due to scarcity or even unavailability of parallel corpora. We propose an SC framework based on variational auto-encoder which enables us to exploit non-parallel corpora. The framework comprises an encoder that learns speaker-independent phonetic representations and a decoder that learns to reconstruct the designated speaker. It removes the requirement of parallel corpora or phonetic alignments to train a spectral conversion system. We report objective and subjective evaluations to validate our proposed method and compare it to SC methods that have access to aligned corpora.Keywords
All Related Versions
This publication has 13 references indexed in Scilit:
- The matching-minimization algorithm, the INCA algorithm and a mathematical framework for voice conversion with unaligned corpora2016
- Mapping frames with DNN-HMM recognizer for non-parallel voice conversion2015
- Non-parallel voice conversion using joint optimization of alignment by temporal context and spectral distortion2014
- Incorporating global variance in the training phase of GMM-based voice conversion2013
- Non-parallel training for voice conversion based on adaptation method2013
- Voice Conversion Based on Weighted Frequency WarpingIEEE Transactions on Audio, Speech, and Language Processing, 2009
- INCA Algorithm for Training Voice Conversion Systems From Nonparallel CorporaIEEE Transactions on Audio, Speech, and Language Processing, 2009
- Text-independent voice conversion based on state mapped codebook2008
- Voice Conversion Based on Maximum-Likelihood Estimation of Spectral Parameter TrajectoryIEEE Transactions on Audio, Speech, and Language Processing, 2007
- Restructuring speech representations using a pitch-adaptive time–frequency smoothing and an instantaneous-frequency-based F0 extraction: Possible role of a repetitive structure in soundsSpeech Communication, 1999