Lombard speech synthesis using long short-term memory recurrent neural networks
- 1 March 2017
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 5505-5509
Abstract
In statistical parametric speech synthesis (SPSS), a few studies have investigated the Lombard effect, specifically by using hidden Markov model (HMM)-based systems. Recently, artificial neural networks have demonstrated promising results in SPSS, specifically by using long short-term memory recurrent neural networks (LSTMs). The Lombard effect, however, has not been studied in the LSTM-based speech synthesis systems. In this study, we propose three methods for Lombard speech adaptation in LSTM-based speech synthesis. In particular, (1) we augment Lombard specific information with the linguistic features as input, (2) scale the hidden activations using the learning hidden unit contributions (LHUC) method, and (3) fine-tune the LSTMs trained on normal speech with a small Lombard speech data. To investigate the effectiveness of the proposed methods, we carry out experiments using small (10 utterances) and large (500 utterances) Lombard speech data. Experimental results confirm the adaptability of the LSTMs, and similarity tests show that the LSTMs can achieve significantly better adaptation performance than the HMMs in both small and large data conditions.Keywords
This publication has 22 references indexed in Scilit:
- Investigating gated recurrent networks for speech synthesis2016
- Unidirectional long short-term memory recurrent neural network with recurrent output layer for low-latency speech synthesis2015
- Enhancing the Intelligibility of Statistically Generated Synthetic Speech by Means of Noise-Independent ModificationsIEEE/ACM Transactions on Audio, Speech, and Language Processing, 2014
- Intelligibility enhancement of HMM-generated speech in additive noise by modifying Mel cepstral coefficients to increase the glimpse proportionComputer Speech & Language, 2014
- Synthesis and perception of breathy, normal, and Lombard speech in the presence of noiseComputer Speech & Language, 2014
- Speaker adaptation of neural network acoustic models using i-vectors2013
- KL-divergence regularized deep neural network adaptation for improved large vocabulary speech recognition2013
- Speech Synthesis Based on Hidden Markov ModelsProceedings of the IEEE, 2013
- Analysis of Speaker Adaptation Algorithms for HMM-Based Speech Synthesis and a Constrained SMAPLR Adaptation AlgorithmIEEE Transactions on Audio, Speech, and Language Processing, 2009
- Improving the Understandability of Speech Synthesis by Modeling Speech in Noise2006