Attention-Based ASR with Lightweight and Dynamic Convolutions
Authors:
- 1 May 2020
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 7034-7038
Abstract
End-to-end (E2E) automatic speech recognition (ASR) with sequence-to-sequence models has gained attention because of its simple model training compared with conventional hidden Markov model based ASR. Recently, several studies report the state-of-the-art E2E ASR results obtained by Transformer. Compared to recurrent neural network (RNN) based E2E models, training of Transformer is more efficient and also achieves better performance on various tasks. However, self-attention used in Transformer requires computation quadratic in its input length. In this paper, we propose to apply lightweight and dynamic convolution to E2E ASR as an alternative architecture to the self-attention to make the computational order linear. We also propose joint training with connectionist temporal classification, convolution on the frequency axis, and combination with self-attention. With these techniques, the proposed architectures achieve better performance than RNN-based E2E model and performance competitive to state-of-the-art Transformer on various ASR benchmarks including noisy/reverberant tasks.Keywords
All Related Versions
This publication has 21 references indexed in Scilit:
- An analysis of environment, microphone and data simulation mismatches in robust speech recognitionComputer Speech & Language, 2017
- Hybrid CTC/Attention Architecture for End-to-End Speech RecognitionIEEE Journal of Selected Topics in Signal Processing, 2017
- A Comparison of Sequence-to-Sequence Models for Speech Recognition2017
- Very deep convolutional networks for end-to-end speech recognition2017
- Towards End-to-End Speech Recognition with Deep Convolutional Neural Networks2016
- Modeling Time-Frequency Patterns with LSTM vs. Convolutional Architectures for LVCSR Tasks2016
- Listen, attend and spell: A neural network for large vocabulary conversational speech recognition2016
- A summary of the REVERB challenge: state-of-the-art and remaining challenges in reverberant speech processing researchEURASIP Journal on Advances in Signal Processing, 2016
- Acoustic Beamforming for Speaker Diarization of MeetingsIEEE Transactions on Audio, Speech, and Language Processing, 2007
- Connectionist temporal classification2006