Group Sparse Representation With WaveNet Vocoder Adaptation for Spectrum and Prosody Conversion
- 11 April 2019
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE/ACM Transactions on Audio, Speech, and Language Processing
- Vol. 27 (6) , 1085-1097
Abstract
The statistical approach to voice conversion typically consists of a feature conversion module followed by a vocoder. So far, the feature conversion studies are mainly focused on the conversion of spectrum. However, speaker identity is also characterized by prosodic features, such as fundamental frequency (F0) and energy contour among others. In this paper, we study the transformation of speaker characteristics both in terms of spectrum and prosody. We propose two novel techniques that effectively use a limited amount of source-target training data and leverage a large general speech corpus to improve the voice conversion quality. First, we study the phonetic sparse representation under the group sparsity mathematical formulation. We use phonetic posteriorgrams (PPGs) together with spectral and prosody features to form tandem feature in the phonetic dictionary. The tandem feature allow us to estimate an activation matrix that is less dependent on source speakers, thus providing a better voice conversion quality. Second, we study the use of WaveNet vocoder that can be trained on general speech corpus from multiple speakers and adapted on target speaker data to improve the vocoding quality. We benefit from the large general speech databases that are used to train the PPG generator, and the WaveNet vocoder. The experiments show that the proposed conversion framework outperforms the traditional spectrum and prosody conversion techniques in both objective and subjective evaluations.Keywords
Funding Information
- Ministry of Education, Singapore AcRF Tier1 NUS Start-up (FY2016)
- SINGA Scholarship under A*STAR Graduate Academy
This publication has 45 references indexed in Scilit:
- Speech Enhancement Under Low SNR Conditions Via Noise Estimation Using Sparse and Low-Rank NMF with Kullback–Leibler DivergenceIEEE/ACM Transactions on Audio, Speech, and Language Processing, 2015
- Activity-mapping non-negative matrix factorization for exemplar-based voice conversion2015
- A multi-level representation of f0 using the continuous wavelet transform and the Discrete Cosine Transform2015
- Speech enhancement using non negative matrix factorization and enhanced NMF2015
- Exemplar-Based Sparse Representation With Residual Compensation for Voice ConversionIEEE/ACM Transactions on Audio, Speech, and Language Processing, 2014
- Semi-supervised noise dictionary adaptation for exemplar-based noise robust speech recognition2014
- A style capturing approach to F0 transformation in voice conversion2013
- Group Sparsity in Nonnegative Matrix Factorization2012
- Speech denoising using nonnegative matrix factorization with priorsInternational Conference on Acoustics, Speech, and Signal Processing (ICASSP), 2008
- Speaker adaptation and voice conversion by codebook mapping1991