Speech Enhancement Guided by Contextual Articulatory Information
Authors:
- Yen-Ju LuYLYen-Ju Lu
- Chia-Yu ChangCCChia-Yu Chang
- Cheng YuCYCheng Yu
- Ching-Feng LiuCLChing-Feng Liu
- Jeih-Weih HungJHJeih-Weih Hung
- Shinji WatanabeSWShinji Watanabe
- Yu TsaoYTYu Tsao
Preprint
Abstract
Previous studies have confirmed that by augmenting acoustic features with the place/manner of articulatory features, the speech enhancement (SE) process can be guided to consider the articulatory properties of the input speech when performing enhancement to attain performance improvements. Thus, the contextual information of articulatory attributes has additional information that can further benefit SE. This study proposed an SE system that improved performance by optimizing contextual articulatory information in enhanced speech through joint training of the SE model with an end-to-end automatic speech recognition (E2E-ASR) model and predicting the sequence of broad phone classes (BPCs) instead of the phoneme/word sequences. We developed two strategies to train the SE system based on BPC-based ASR: multi-task learning and deep-feature training strategies. Experimental results from speech denoising, speech dereverberation, and impaired speech enhancement tasks confirmed that the contextual articulatory information facilitates the SE system to improve enhancement results. Moreover, in contrast to another SE system trained with monophonic ASR, the BPC-based ASR (providing contextual articulatory information) can achieve superior SE performance at different signal-to-noise ratio (SNR) levels.Keywords
All Related Versions
This publication has 27 references indexed in Scilit:
- Convolutional fusion network for monaural speech enhancementNeural Networks, 2021
- Real-time Speech Enhancement Using an Efficient Convolutional Recurrent Network for Dual-microphone Mobile Phones in Close-talk Scenarios2019
- Improving Mandarin Tone Recognition Based on DNN by Combining Acoustic and Articulatory Features Using Extended Recognition NetworksJournal of Signal Processing Systems, 2018
- Hybrid CTC/Attention Architecture for End-to-End Speech RecognitionIEEE Journal of Selected Topics in Signal Processing, 2017
- Deep learning reinvents the hearing aidIEEE Spectrum, 2017
- Phase-sensitive and recognition-boosted speech separation using deep recurrent neural networks2015
- Joint training of front-end and back-end deep neural networks for robust speech recognition2015
- An artificial neural network approach to automatic speech processingNeurocomputing, 2014
- Wiener filtering based speech enhancement with Weighted Denoising Auto-encoder and noise classificationSpeech Communication, 2014
- Exploiting deep neural networks for detection-based speech recognitionNeurocomputing, 2013