Predicting Expressive Speaking Style from Text in End-To-End Speech Synthesis
Authors:
- 1 December 2018
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 595-602
Abstract
Global Style Tokens (GSTs) are a recently-proposed method to learn latent disentangled representations of high-dimensional data. GSTs can be used within Tacotron, a state-of-the-art end-to-end text-to-speech synthesis system, to uncover expressive factors of variation in speaking style. In this work, we introduce the Text-Predicting Global Style Token (TP-GST) architecture, which treats GST combination weights or style embeddings as “virtual” speaking style labels within Tacotron. TP-GST learns to predict stylistic renderings from text alone, requiring neither explicit labels during training, nor auxiliary inputs for inference. We show that, when trained on an expressive speech dataset, our system can render text with more pitch and energy variation than two state-of-the-art baseline models. We further demonstrate that TP-GSTs can synthesize speech with background noise removed, and corroborate these analyses with positive results on human-rated listener preference audiobook tasks. Finally, we demonstrate that multi-speaker TP-GST models successfully factorize speaker identity and speaking style. We provide a website with audio samples 1 for each of our findings.Keywords
All Related Versions
This publication has 11 references indexed in Scilit:
- Expressive Speech Synthesis via Modeling Expressions with Variational Autoencoder2018
- Investigating different representations for modeling and controlling multiple emotions in DNN-based speech synthesisSpeech Communication, 2018
- Semisupervised Autoencoders for Speech Emotion RecognitionIEEE/ACM Transactions on Audio, Speech, and Language Processing, 2017
- Tacotron: Towards End-to-End Speech Synthesis2017
- Learning utterance-level representations for speech emotion and age/gender recognition using deep neural networks2017
- High-level feature representation using recurrent neural network for speech emotion recognition2015
- SLAM: Automatic Stylization and Labelling of Speech Melody2014
- Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation2014
- Front-End Factor Analysis for Speaker VerificationIEEE Transactions on Audio, Speech, and Language Processing, 2010
- Landmark detection for distinctive feature-based speech recognitionThe Journal of the Acoustical Society of America, 1996