E-DGAN: An Encoder-Decoder Generative Adversarial Network Based Method for Pathological to Normal Voice Conversion
- 24 January 2023
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Journal of Biomedical and Health Informatics
- Vol. 27 (5) , 2489-2500
Abstract
In recent years, more and more people suffer from voice-related diseases. Given the limitations of current pathological speech conversion methods, that is, a method can only convert a single kind of pathological voice. In this study, we propose a novel Encoder-Decoder Generative Adversarial Network (E-DGAN) to generate personalized speech for pathological to normal voice conversion, which is suitable for multiple kinds of pathological voices. Our proposed method can also solve the problem of improving the intelligibility and personalizing custom speech of pathological voices. Feature extraction is performed using a mel filter bank. The conversion network is an encoder-decoder structure, which is used to convert the mel spectrogram of pathological voices to the mel spectrogram of normal voices. After being converted by the residual conversion network, the personalized normal speech is synthesized by the neural vocoder. In addition, we propose a subjective evaluation metric named “content similarity” to evaluate the consistency between the converted pathological voice content and the reference content. The Saarbrücken Voice Database (SVD) is used to verify the proposed method. The intelligibility and content similarity of pathological voices are increased by 18.67% and 2.60%, respectively. Besides, an intuitive analysis based on a spectrogram was done and a significant improvement was achieved. The results show that our proposed method can improve the intelligibility of pathological voices and personalize the conversion of pathological voices into the normal voices of 20 different speakers. Our proposed method is compared with five other pathological voice conversion methods, and our proposed method has the best evaluation results.Keywords
Funding Information
- National Natural Science Foundation of China (41971375)
This publication has 43 references indexed in Scilit:
- Voice Conversion Using Sequence-to-Sequence Learning of Context Posterior Probabilities2017
- Finite Element Synthesis of Diphthongs Using Tuned Two-Dimensional Vocal TractsIEEE/ACM Transactions on Audio, Speech, and Language Processing, 2017
- Voice Pathology Detection and Classification Using Auto-Correlation and Entropy Features in Different Frequency RegionsIEEE Access, 2017
- The NU-NAIST Voice Conversion System for the Voice Conversion Challenge 20162016
- Automatic speech recognition for acoustical analysis and assessment of cantonese pathological voice and speech2016
- Semi-supervised noise dictionary adaptation for exemplar-based noise robust speech recognition2014
- An Improved Time Domain Pitch Detection Algorithm for Pathological VoiceAmerican Journal of Applied Sciences, 2012
- Speech denoising using nonnegative matrix factorization with priorsInternational Conference on Acoustics, Speech, and Signal Processing (ICASSP), 2008
- Methods for formant extraction in speech of patients after total laryngectomyBiomedical Signal Processing and Control, 2006
- Communication, functional disorders and lifestyle changes after total laryngectomyClinical Otolaryngology, 1994