Streaming Chunk-Aware Multihead Attention for Online End-to-End Speech Recognition
Preprint
Abstract
Recently, streaming end-to-end automatic speech recognition (E2E-ASR) has gained more and more attention. Many efforts have been paid to turn the non-streaming attention-based E2E-ASR system into streaming architecture. In this work, we propose a novel online E2E-ASR system by using Streaming Chunk-Aware Multihead Attention(SCAMA) and a latency control memory equipped self-attention network (LC-SAN-M). LC-SAN-M uses chunk-level input to control the latency of encoder. As to SCAMA, a jointly trained predictor is used to control the output of encoder when feeding to decoder, which enables decoder to generate output in streaming manner. Experimental results on the open 170-hour AISHELL-1 and an industrial-level 20000-hour Mandarin speech recognition tasks show that our approach can significantly outperform the MoChA-based baseline system under comparable setup. On the AISHELL-1 task, our proposed method achieves a character error rate (CER) of 7.39%, to the best of our knowledge, which is the best published performance for online ASR.Keywords
All Related Versions
This publication has 9 references indexed in Scilit:
- SAN-M: Memory Equipped Self-Attention for End-to-End Speech Recognition2020
- Streaming Automatic Speech Recognition with the Transformer Model2020
- A Comparative Study on Transformer vs RNN in Speech Applications2019
- Very Deep Self-Attention Networks for End-to-End Speech Recognition2019
- An Online Attention-Based Model for Speech Recognition2019
- SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognition2019
- Gaussian Prediction Based Attention for Online End-to-End Speech Recognition2017
- Nonrecurrent Neural Structure for Long-Term DependenceIEEE/ACM Transactions on Audio, Speech, and Language Processing, 2017
- Fast and accurate recurrent neural network acoustic models for speech recognition2015