Improving Speech Enhancement Performance by Leveraging Contextual Broad Phonetic Class Information
Open Access
- 27 June 2023
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE/ACM Transactions on Audio, Speech, and Language Processing
- Vol. 31 (23299290) , 2738-2750
Abstract
Previous studies have confirmed that by augmenting acoustic features with the place/manner of articulatory features, the speech enhancement (SE) process can be guided to consider the broad phonetic properties of the input speech when performing enhancement to attain performance improvements. In this paper, we explore the contextual information of articulatory attributes as additional information to further benefit SE. More specifically, we propose to improve the SE performance by leveraging losses from an end-to-end automatic speech recognition (E2E-ASR) model that predicts the sequence of broad phonetic classes (BPCs). We also developed multi-objective training with ASR and perceptual losses to train the SE system based on a BPC-based E2E-ASR. Experimental results from speech denoising, speech dereverberation, and impaired speech enhancement tasks confirmed that contextual BPC information improves SE performance. Moreover, the SE model trained with the BPC-based E2E-ASR outperforms that with the phoneme-based E2E-ASR. The results suggest that objectives with misclassification of phonemes by the ASR system may lead to imperfect feedback, and BPC could be a potentially better choice. Finally, it is noted that combining the most-confusable phonetic targets into the same BPC when calculating the additional objective can effectively improve the SE performance.Keywords
All Related Versions
This publication has 51 references indexed in Scilit:
- Speech Enhancement with LSTM Recurrent Neural Networks and its Application to Noise-Robust ASR2015
- Phase-sensitive and recognition-boosted speech separation using deep recurrent neural networks2015
- Learning Spectral Mapping for Speech Dereverberation and DenoisingIEEE/ACM Transactions on Audio, Speech, and Language Processing, 2015
- A Regression Approach to Speech Enhancement Based on Deep Neural NetworksIEEE/ACM Transactions on Audio, Speech, and Language Processing, 2014
- An artificial neural network approach to automatic speech processingNeurocomputing, 2014
- On Training Targets for Supervised Speech SeparationIEEE/ACM Transactions on Audio, Speech, and Language Processing, 2014
- Wiener filtering based speech enhancement with Weighted Denoising Auto-encoder and noise classificationSpeech Communication, 2014
- Exploiting deep neural networks for detection-based speech recognitionNeurocomputing, 2013
- Broad phonetic class definition driven by phone confusionsEURASIP Journal on Advances in Signal Processing, 2012
- New speech enhancement techniques for low bit rate speech coding1999