Structured Sparse Spectral Transforms and Structural Measures for Voice Conversion
- 27 July 2018
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE/ACM Transactions on Audio, Speech, and Language Processing
- Vol. 26 (12) , 2267-2276
Abstract
We investigate a structured sparse spectral transform method for voice conversion (VC) to perform frequency warping and spectral shaping simultaneously on high-dimensional (D) STRAIGHT spectra. Learning a large transform matrix for high-D data often results in an overfit matrix with low sparsity, which leads to muffled speech in VC. We address this problem by using the frequency-warping characteristic of a source-target speaker pair to define a region of support (ROS) in a transform matrix, and further optimize it by nonnegative matrix factorization (NMF) to obtain structured sparse transform. We also investigate structural measures of spectral and temporal covariance and variance at different scales for assessing VC speech quality. Our experiments on ARCTIC dataset of 12 speaker pairs show that embedding the ROS in spectral transforms offers flexibility in tradeoffs between spectral distortion and structure preservation, and the structural measures provide quantitatively reasonable results on converted speech. Our subjective listening tests show that the proposed VC method achieves a mean opinion score of “very good” relative to natural speech, and in comparison with three other VC methods, it is the most preferred one in naturalness and in voice similarity to target speakers.Keywords
Funding Information
- National Institutes of Health (R15, DC016383)
This publication has 26 references indexed in Scilit:
- An Exemplar-Based Approach to Frequency Warping for Voice ConversionIEEE/ACM Transactions on Audio, Speech, and Language Processing, 2017
- The Voice Conversion Challenge 20162016
- Exemplar-Based Sparse Representation With Residual Compensation for Voice ConversionIEEE/ACM Transactions on Audio, Speech, and Language Processing, 2014
- Parametric Voice Conversion Based on Bilinear Frequency Warping Plus Amplitude ScalingIEEE Transactions on Audio, Speech, and Language Processing, 2012
- An Algorithm for Intelligibility Prediction of Time–Frequency Weighted Noisy SpeechIEEE Transactions on Audio, Speech, and Language Processing, 2011
- Continuous Stochastic Feature Mapping Based on Trajectory HMMsIEEE Transactions on Audio, Speech, and Language Processing, 2010
- Voice Conversion Based on Weighted Frequency WarpingIEEE Transactions on Audio, Speech, and Language Processing, 2009
- Sparse Principal Component AnalysisJournal of Computational and Graphical Statistics, 2006
- Analysis of speech-based speech transmission index methods with implications for nonlinear operationsThe Journal of the Acoustical Society of America, 2004
- Image quality assessment: from error visibility to structural similarityIEEE Transactions on Image Processing, 2004