Large-Scale Unsupervised Pre-Training for End-to-End Spoken Language Understanding
Authors:
- 1 May 2020
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 7999-8003
Abstract
End-to-end Spoken Language Understanding (SLU) is proposed to infer the semantic meaning directly from audio features without intermediate text representation. In this paper, we explore unsupervised pre-training for End-to-end SLU models by learning representations from large-scale raw audios. The pre-trained model preserves semantic features which benefit the downstream SLU tasks as the learned model weights are further fine-tuned on the task specific training data. Our approach out-perform the state-of-the-art end-to-end SLU system with over 18.33% error reduction.Keywords
This publication has 14 references indexed in Scilit:
- From Audio to Semantics: Approaches to End-to-End Spoken Language Understanding2018
- Spoken Language Understanding without Speech Recognition2018
- Deep-FSMN for Large Vocabulary Continuous Speech Recognition2018
- Deep Contextualized Word Representations2018
- VoxCeleb: A Large-Scale Speaker Identification Dataset2017
- Librispeech: An ASR corpus based on public domain audio books2015
- Using Recurrent Neural Networks for Slot Filling in Spoken Language UnderstandingIEEE/ACM Transactions on Audio, Speech, and Language Processing, 2014
- Contextual domain classification in spoken language understanding systems using recurrent neural network2014
- Towards deeper understanding: Deep convex networks for semantic utterance classification2012
- How may I help you?Speech Communication, 1997