Sequence-to-Sequence Singing Synthesis Using the Feed-Forward Transformer
Authors:
- 1 May 2020
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 7229-7233
Abstract
We propose a sequence-to-sequence singing synthesizer, which avoids the need for training data with pre-aligned phonetic and acoustic features. Rather than the more common approach of a content-based attention mechanism combined with an autoregressive decoder, we use a different mechanism suitable for feed-forward synthesis. Given that phonetic timings in singing are highly constrained by the musical score, we derive an approximate initial alignment with the help of a simple duration model. Then, using a decoder based on a feed-forward variant of the Transformer model, a series of self-attention and convolutional layers refines the result of the initial alignment to reach the target acoustic features. Advantages of this approach include faster inference and avoiding the exposure bias issues that affect autoregressive models trained by teacher forcing. We evaluate the effectiveness of this model compared to an autoregressive baseline, the importance of self-attention, and the importance of the accuracy of the duration model.Keywords
All Related Versions
This publication has 12 references indexed in Scilit:
- Adversarially Trained End-to-End Korean Singing Voice Synthesis System2019
- Singing Voice Synthesis Using Deep Autoregressive Neural Networks for Acoustic Modeling2019
- Neural Speech Synthesis with Transformer NetworkProceedings of the AAAI Conference on Artificial Intelligence, 2019
- Singing Voice Synthesis Based on Generative Adversarial Networks2019
- Neural Source-filter-based Waveform Model for Statistical Parametric Speech Synthesis2019
- Self-Attentional Acoustic Models2018
- Natural TTS Synthesis by Conditioning Wavenet on MEL Spectrogram Predictions2018
- A Neural Parametric Singing Synthesizer Modeling Timbre and Expression from Natural SongsApplied Sciences, 2017
- Speaker-Dependent WaveNet Vocoder2017
- WORLD: A Vocoder-Based High-Quality Speech Synthesis System for Real-Time ApplicationsIEICE Transactions on Information and Systems, 2016