Accelerating RNN-T Training and Inference Using CTC Guidance
Authors:
- 4 June 2023
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 1-5
Abstract
We propose a novel method to accelerate training and inference process of recurrent neural network transducer (RNN-T) based on the guidance from a co-trained connectionist temporal classification (CTC) model. We made a key assumption that if an encoder embedding frame is classified as a blank frame by the CTC model, it is likely that this frame will be aligned to blank for all the partial alignments or hypotheses in RNN-T and it can be discarded from the decoder input. We also show that this frame reduction operation can be applied in the middle of the encoder, which result in significant speed up for the training and inference in RNN-T. We further show that the CTC alignment, a by-product of the CTC decoder, can also be used to perform lattice reduction for RNN-T during training. Our method is evaluated on the Librispeech and SpeechStew tasks. We demonstrate that the proposed method is able to accelerate the RNN-T inference by 2.2 times with similar or slightly better word error rates (WER).Keywords
All Related Versions
This publication has 20 references indexed in Scilit:
- A Comparative Study on Transformer vs RNN in Speech Applications2019
- Streaming End-to-end Speech Recognition for Mobile Devices2019
- Phone Synchronous Speech Recognition With CTC LatticesIEEE/ACM Transactions on Audio, Speech, and Language Processing, 2016
- Lower Frame Rate Neural Network Acoustic Models2016
- Learning acoustic frame labeling for speech recognition with recurrent neural networks2015
- Librispeech: An ASR corpus based on public domain audio books2015
- Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation2014
- Speech recognition with deep recurrent neural networks2013
- Japanese and Korean voice search2012
- Connectionist temporal classification2006