FastEmit: Low-Latency Streaming ASR with Sequence-Level Emission Regularization
Authors:
- 6 June 2021
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 6004-6008
Abstract
Streaming automatic speech recognition (ASR) aims to emit each hypothesized word as quickly and accurately as possible. However, emitting fast without degrading quality, as measured by word error rate (WER), is highly challenging. Existing approaches including Early and Late Penalties [1] and Constrained Alignments [2], [3] penalize emission delay by manipulating per-token or per-frame probability prediction in sequence transducer models [4]. While being successful in reducing delay, these approaches suffer from significant accuracy regression and also require additional word alignment information from an existing model. In this work, we propose a sequence-level emission regularization method, named FastEmit, that applies latency regularization directly on per-sequence probability in training transducer models, and does not require any alignment. We demonstrate that FastEmit is more suitable to the sequence-level optimization of transducer models [4] for streaming ASR by applying it on various end-to-end streaming ASR networks including RNN-Transducer [5], Transformer-Transducer [6], [7], ConvNet-Transducer [8] and Conformer-Transducer [9]. We achieve 150 ~ 300ms latency reduction with significantly better accuracy over previous techniques on a Voice Search test set. FastEmit also improves streaming ASR accuracy from 4.4%/8.9% to 3.1%/7.5% WER, meanwhile reduces 90th percentile latency from 210ms to only 30ms on LibriSpeech.Keywords
All Related Versions
This publication has 20 references indexed in Scilit:
- A Comparison of End-to-End Models for Long-Form Speech Recognition2019
- Two-Pass End-to-End Speech Recognition2019
- SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognition2019
- Streaming End-to-end Speech Recognition for Mobile Devices2019
- Toward Domain-Invariant Speech Recognition via Large Scale Training2018
- Minimum Word Error Rate Training for Attention-Based Sequence-to-Sequence Models2018
- Multi-Dialect Speech Recognition with a Single Sequence-to-Sequence Model2018
- Sampled Connectionist Temporal Classification2018
- Endpoint Detection Using Grid Long Short-Term Memory Networks for Streaming Speech Recognition2017
- Librispeech: An ASR corpus based on public domain audio books2015