Towards Expressive Speaking Style Modelling with Hierarchical Context Information for Mandarin Speech Synthesis
Authors:
- 23 May 2022
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 7922-7926
Abstract
Previous works on expressive speech synthesis mainly focus on current sentence. The context in adjacent sentences is neglected, resulting in inflexible speaking style for the same text, which lacks speech variations. In this paper, we propose a hierarchical framework to model speaking style from context. A hierarchical context encoder is proposed to explore a wider range of contextual information considering structural relationship in context, including inter-phrase and inter-sentence relations. Moreover, to encourage this encoder to learn style representation better, we introduce a novel training strategy with knowledge distillation, which provides the target for encoder training. Both objective and subjective evaluations on a Mandarin lecture dataset demonstrate that the proposed method can significantly improve the naturalness and expressiveness of the synthesized speech 1 .Keywords
All Related Versions
Funding Information
- National Natural Science Foundation of China
This publication has 11 references indexed in Scilit:
- Extracting and Predicting Word-Level Style Variations for Speech SynthesisIEEE/ACM Transactions on Audio, Speech, and Language Processing, 2021
- Deep Learning--based Text ClassificationACM Computing Surveys, 2021
- Improving Prosody with Linguistic and Bert Derived Features in Multi-Speaker Based Mandarin Chinese Neural TTS2020
- Evaluating Long-form Text-to-Speech: Comparing the Ratings of Sentences and Paragraphs2019
- Pre-Trained Text Embeddings for Enhanced Text-to-Speech Synthesis2019
- Learning Latent Representations for Style Control and Transfer in End-to-end Speech Synthesis2019
- Predicting Expressive Speaking Style from Text in End-To-End Speech Synthesis2018
- Natural TTS Synthesis by Conditioning Wavenet on MEL Spectrogram Predictions2018
- Montreal Forced Aligner: Trainable Text-Speech Alignment Using Kaldi2017
- Hierarchical Attention Networks for Document Classification2016