Confidence Estimation and Deletion Prediction Using Bidirectional Recurrent Neural Networks
Authors:
- 1 December 2018
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 204-211
Abstract
The standard approach to assess reliability of automatic speech transcriptions is through the use of confidence scores. If accurate, these scores provide a flexible mechanism to flag transcription errors for upstream and downstream applications. One challenging type of errors that recognisers make are deletions. These errors are not accounted for by the standard confidence estimation schemes and are hard to rectify in the upstream and downstream processing. High deletion rates are prominent in limited resource and highly mismatched training/testing conditions studied under IARPA Babel and Material programs. This paper looks at the use of bidirectional recurrent neural networks to yield confidence estimates in predicted as well as deleted words. Several simple schemes are examined for combination. To assess usefulness of this approach, the combined confidence score is examined for untranscribed data selection that favours transcriptions with lower deletion errors. Experiments are conducted using IARPA Babel/Material program languages.Keywords
This publication has 25 references indexed in Scilit:
- Estimating confidence scores on ASR results using recurrent neural networks2015
- An Introduction to Conditional Random FieldsFoundations and Trends in Machine Learning, 2012
- Joint-sequence models for grapheme-to-phoneme conversionSpeech Communication, 2008
- The relationship between Precision-Recall and ROC curves2006
- Confidence measures for speech recognition: A surveySpeech Communication, 2005
- Improving broadcast news transcription by lightly supervised discriminative training2004
- Neural-network based measures of confidence for word recognition2002
- A probabilistic approach to confidence estimation and evaluation2002
- Confidence measures for spontaneous speech recognition2002
- Using word probabilities as confidence measures1998