Leveraging Unpaired Text Data for Training End-To-End Speech-to-Intent Systems
Authors:
- 1 May 2020
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 7984-7988
Abstract
Training an end-to-end (E2E) neural network speech-to-intent (S2I) system that directly extracts intents from speech requires large amounts of intent-labeled speech data, which is time consuming and expensive to collect. Initializing the S2I model with an ASR model trained on copious speech data can alleviate data sparsity. In this paper, we attempt to leverage NLU text resources. We implemented a CTC-based S2I system that matches the performance of a state-of-the-art, traditional cascaded SLU system. We performed controlled experiments with varying amounts of speech and text training data. When only a tenth of the original data is available, intent classification accuracy degrades by 7.6% absolute. Assuming we have additional text-to-intent data (without speech) available, we investigated two techniques to improve the S2I system: (1) transfer learning, in which acoustic embeddings for intent classification are tied to fine-tuned BERT text embeddings; and (2) data augmentation, in which the text-to-intent data is converted into speech-to-intent data using a multi-speaker text-to-speech system. The proposed approaches recover 80% of performance lost due to using limited intent-labeled speech.Keywords
All Related Versions
This publication has 19 references indexed in Scilit:
- LPCNET: Improving Neural Speech Synthesis through Linear Prediction2019
- Sequence Noise Injected Training for End-to-end Speech Recognition2019
- End-To-End Named Entity And Semantic Concept Extraction From Speech2018
- From Speech Signals to Semantics — Tagging Performance at Acoustic, Phonetic and Word Levels2018
- Spoken Language Understanding without Speech Recognition2018
- Towards End-to-end Spoken Language Understanding2018
- Exploring ASR-free end-to-end modeling to improve spoken language understanding in a cloud-based dialog system2017
- An Integrative and Discriminative Technique for Spoken Utterance ClassificationIEEE Transactions on Audio, Speech, and Language Processing, 2008
- Language Model Estimation for Optimizing End-to-end Performance of a Natural Language Call Routing System2006
- Connectionist temporal classification2006