Adapting and controlling DNN-based speech synthesis using input codes
Authors:
- 1 March 2017
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 4905-4909
Abstract
Methods for adapting and controlling the characteristics of output speech are important topics in speech synthesis. In this work, we investigated the performance of DNN-based text-to-speech systems that in parallel to conventional text input also take speaker, gender, and age codes as inputs, in order to 1) perform multi-speaker synthesis, 2) perform speaker adaptation using small amounts of target-speaker adaptation data, and 3) modify synthetic speech characteristics based on the input codes. Using a large-scale, studio-quality speech corpus with 135 speakers of both genders and ages between tens and eighties, we performed three experiments: 1) First, we used a subset of speakers to construct a DNN-based, multi-speaker acoustic model with speaker codes. 2) Next, we performed speaker adaptation by estimating code vectors for new speakers via backpropagation from a small amount of adaptation material. 3) Finally, we experimented with manually manipulating input code vectors to alter the gender and/or age characteristics of the synthesised speech. Experimental results show that high-performance multi-speaker models can be constructed using the proposed code vectors with a variety of encoding schemes, and that adaptation and manipulation can be performed effectively using the codes.Keywords
This publication has 15 references indexed in Scilit:
- Merlin: An Open Source Neural Network Speech Synthesis System2016
- A deep auto-encoder based low-dimensional feature extraction from FFT spectral envelopes for statistical parametric speech synthesis2016
- From HMMS to DNNS: Where do the improvements come from?2016
- Deep Learning for Acoustic Modeling in Parametric Speech Generation: A systematic review of existing techniques and future trendsIEEE Signal Processing Magazine, 2015
- Learning hidden unit contributions for unsupervised speaker adaptation of neural network acoustic models2014
- Fast Adaptation of Deep Neural Network Based on Discriminant Codes for Speech RecognitionIEEE/ACM Transactions on Audio, Speech, and Language Processing, 2014
- A Style Control Technique for HMM-Based Expressive Speech SynthesisIEICE Transactions on Information and Systems, 2007
- Average-Voice-Based Speech Synthesis Using HSMM-Based Speaker Adaptation and Adaptive TrainingIEICE Transactions on Information and Systems, 2007
- STRAIGHT, exploitation of the other aspect of VOCODER: Perceptually isomorphic decomposition of speech soundsAcoustical Science and Technology, 2006
- Speech Synthesis with Various Emotional Expressions and Speaking Styles by Style Interpolation and MorphingIEICE Transactions on Information and Systems, 2005