Low-Resource Speech Synthesis with Speaker-Aware Embedding
Authors:
- 11 December 2022
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 235-239
Abstract
Speech synthesis has been successfully exploited for mapping from text sequence to speech waveform where high-resource languages have been well studied and learned from a large amount of text-speech paired data in public-domain corpora. However, developing speech synthesis under low-resource languages is challenging for speech communication in local regions since the collection of training data is expensive. In particular, the speaker-aware speech generation under low-resource settings is crucial in real world. Such a problem is increasingly difficult in case of very limited speaker-specific data. This paper presents a speaker-aware speech synthesis under low-resource settings based on an encoder-decoder framework by using transformer. Knowledge transfer is performed by incorporating a speaker-aware embedding through first learning a pretrained transformer from multi-speaker data of a low-populated spoken language and then fine-tuning the transformer to a target speaker with very limited speaker-specific embeddings. Experiments on low-resource Taiwanese speech synthesis are evaluated to show the merit of speaker-aware transformer in terms of Mel cepstral distortion and mean opinion score.Keywords
This publication has 20 references indexed in Scilit:
- LibriTTS: A Corpus Derived from LibriSpeech for Text-to-Speech2019
- Neural Speech Synthesis with Transformer NetworkProceedings of the AAAI Conference on Artificial Intelligence, 2019
- ESPnet: End-to-End Speech Processing Toolkit2018
- Multisource I-Vectors Domain Adaptation Using Maximum Mean Discrepancy Based AutoencodersIEEE/ACM Transactions on Audio, Speech, and Language Processing, 2018
- Natural TTS Synthesis by Conditioning Wavenet on MEL Spectrogram Predictions2018
- X-Vectors: Robust DNN Embeddings for Speaker Recognition2018
- Deep neural networks for small footprint text-dependent speaker verification2014
- Introduction to the Special Section on Deep Learning for Speech and Language ProcessingIEEE Transactions on Audio, Speech, and Language Processing, 2011
- Front-End Factor Analysis for Speaker VerificationIEEE Transactions on Audio, Speech, and Language Processing, 2010
- Mel-cepstral distance measure for objective speech quality assessment2002