Efficient lattice rescoring using recurrent neural network language models
Authors:
- 1 May 2014
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- No. 15206149,p. 4908-4912
Abstract
Recurrent neural network language models (RNNLM) have become an increasingly popular choice for state-of-the-art speech recognition systems due to their inherently strong generalization performance. As these models use a vector representation of complete history contexts, RNNLMs are normally used to rescore N-best lists. Motivated by their intrinsic characteristics, two novel lattice rescoring methods for RNNLMs are investigated in this paper. The first uses an n-gram style clustering of history contexts. The second approach directly exploits the distance measure between hidden history vectors. Both methods produced 1-best performance comparable with a 10k-best rescoring baseline RNNLM system on a large vocabulary conversational telephone speech recognition task. Significant lattice size compression of over 70% and consistent improvements after confusion network (CN) decoding were also obtained over the N-best rescoring approach.Keywords
This publication has 16 references indexed in Scilit:
- Comparison of feedforward and recurrent neural network language models2013
- Use of contexts in language model interpolation and adaptationComputer Speech & Language, 2013
- Structured Output Layer Neural Network Language Models for Speech RecognitionIEEE Transactions on Audio, Speech, and Language Processing, 2012
- Extensions of recurrent neural network language model2011
- Continuous space language modelsComputer Speech & Language, 2007
- Getting more mileage from web text sources for conversational speech language modeling using class-dependent mixtures2003
- A variable-length category-based n-gram language model2002
- Dynamic programming search for continuous speech recognitionIEEE Signal Processing Magazine, 1999
- A one pass decoder design for large vocabulary recognition1994
- Learning representations by back-propagating errorsNature, 1986