An investigation to transplant emotional expressions in DNN-based TTS synthesis
Authors:
- 1 December 2017
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 1253-1258
Abstract
In this paper, we investigate deep neural network (DNN) architectures to transplant emotional expressions to improve the expressiveness of DNN-based text-to-speech (TTS) synthesis. DNN is expected to have potential power in mapping between linguistic information and acoustic features. From multispeaker and/or multi-language perspectives, several types of DNN architecture have been proposed and have shown good performances. We tried to expand the idea to transplant emotion, constructing shared emotion-dependent mappings. The following three types of DNN architecture are examined; (1) the parallel model (PM) with an output layer consisting of both speaker- dependent layers and emotion-dependent layers, (2) the serial model (SM) with an output layer consisting of emotion-dependent layers preceded by speaker-dependent hidden layers, (3) the auxiliary input model (AIM) with an input layer consisting of emotion and speaker IDs as well as linguistics feature vectors. The DNNs were trained using neutral speech uttered by 24 speakers, and sad speech and joyful speech uttered by 3 speakers from those 24 speakers. In terms of unseen emotional synthesis, subjective evaluation tests showed that the PM performs much better than the SM and slightly better than the AIM. In addition, this test showed that the SM is the best of the three models when training data includes emotional speech uttered by the target speaker.Keywords
This publication has 13 references indexed in Scilit:
- Adapting and controlling DNN-based speech synthesis using input codes2017
- Multi-Language Multi-Speaker Acoustic Modeling for LSTM-RNN Based Statistical Parametric Speech Synthesis2016
- An Investigation of DNN-Based Speech Synthesis Using Speaker Codes2016
- Multi-speaker modeling and speaker adaptation for DNN-based TTS synthesis2015
- Deep neural networks employing Multi-Task Learning and stacked bottleneck features for speech synthesis2015
- Speaker-independent style conversion for HMM-based expressive speech synthesis2013
- Analysis of Speaker Adaptation Algorithms for HMM-Based Speech Synthesis and a Constrained SMAPLR Adaptation AlgorithmIEEE Transactions on Audio, Speech, and Language Processing, 2009
- A Style Control Technique for HMM-Based Expressive Speech SynthesisIEICE Transactions on Information and Systems, 2007
- Speech Synthesis with Various Emotional Expressions and Speaking Styles by Style Interpolation and MorphingIEICE Transactions on Information and Systems, 2005
- Restructuring speech representations using a pitch-adaptive time–frequency smoothing and an instantaneous-frequency-based F0 extraction: Possible role of a repetitive structure in soundsSpeech Communication, 1999