Reinforcement Learning Based Speech Enhancement for Robust Speech Recognition
Authors:
- 1 May 2019
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 6750-6754
Abstract
Conventional deep neural network (DNN)-based speech enhancement (SE) approaches aim to minimize the mean square error (MSE) between enhanced speech and clean reference. The MSE-optimized model may not directly improve the performance of an automatic speech recognition (ASR) system. If the target is to minimize the recognition error, the recognition results should be used to design the objective function for optimizing the SE model. However, the structure of an ASR system, which consists of multiple units, such as acoustic and language models, is usually complex and not differentiable. In this study, we propose to adopt the reinforcement learning (RL) algorithm to optimize the SE model based on the recognition results. We evaluated the proposed RL-based SE system on the Mandarin Chinese broadcast news corpus (MATBN). Experimental results demonstrate that the proposed SE system can effectively improve the ASR results with a notable 12:40% and 19:23% error rate reductions for signal to noise ratio (SNR) at 0 dB and 5 dB conditions, respectively.Keywords
All Related Versions
This publication has 21 references indexed in Scilit:
- Speech enhancement for robust automatic speech recognition: Evaluation using a baseline system and instrumental measuresComputer Speech & Language, 2017
- DNN-based source enhancement self-optimized by reinforcement learning using sound quality measurements2017
- Employing median filtering to enhance the complex-valued acoustic spectrograms in modulation domain for noise-robust speech recognition2016
- A Joint Training Framework for Robust Automatic Speech RecognitionIEEE/ACM Transactions on Audio, Speech, and Language Processing, 2016
- Exemplar-based speech enhancement for deep neural network based automatic speech recognition2015
- Human-level control through deep reinforcement learningNature, 2015
- Environmental robustness in automatic speech recognition2002
- Reinforcement learning is direct adaptive optimal controlIEEE Control Systems, 1992
- Estimation of probabilities from sparse data for the language model component of a speech recognizerIEEE Transactions on Acoustics, Speech, and Signal Processing, 1987
- Speech enhancement using a soft-decision noise suppression filterIEEE Transactions on Acoustics, Speech, and Signal Processing, 1980