Speech Enhancement Guided by Contextual Articulatory Information
Authors:
- Yen-Ju LuYLYen-Ju Lu
- Chia-Yu ChangCCChia-Yu Chang
- Cheng YuCYCheng Yu
- Jeih-Weih HungJHJeih-Weih Hung
- Shinji WatanabeSWShinji Watanabe
- Yu TsaoYTYu Tsao
Preprint
Abstract
Previous studies have confirmed the effectiveness of leveraging articulatory information to attain improved speech enhancement (SE) performance. By augmenting the original acoustic features with the place/manner of articulatory features, the SE process can be guided to consider the articulatory properties of the input speech when performing enhancement. Hence, we believe that the contextual information of articulatory attributes should include useful information and can further benefit SE in different languages. In this study, we propose an SE system that improves its performance through optimizing the contextual articulatory information in enhanced speech for both English and Mandarin. We optimize the contextual articulatory information through joint-train the SE model with an end-to-end automatic speech recognition (E2E ASR) model, predicting the sequence of broad phone classes (BPC) instead of the word sequences. Meanwhile, two training strategies are developed to train the SE system based on the BPC-based ASR: multitask-learning and deep-feature training strategies. Experimental results on the TIMIT and TMHINT dataset confirm that the contextual articulatory information facilitates an SE system in achieving better results than the traditional Acoustic Model(AM). Moreover, in contrast to another SE system that is trained with monophonic ASR, the BPC-based ASR (providing contextual articulatory information) can improve the SE performance more effectively under different signal-to-noise ratios(SNR).Keywords
All Related Versions
This publication has 25 references indexed in Scilit:
- Real-time Speech Enhancement Using an Efficient Convolutional Recurrent Network for Dual-microphone Mobile Phones in Close-talk Scenarios2019
- Improving Mandarin Tone Recognition Based on DNN by Combining Acoustic and Articulatory Features Using Extended Recognition NetworksJournal of Signal Processing Systems, 2018
- Deep learning reinvents the hearing aidIEEE Spectrum, 2017
- Phase-sensitive and recognition-boosted speech separation using deep recurrent neural networks2015
- Joint training of front-end and back-end deep neural networks for robust speech recognition2015
- An artificial neural network approach to automatic speech processingNeurocomputing, 2014
- Wiener filtering based speech enhancement with Weighted Denoising Auto-encoder and noise classificationSpeech Communication, 2014
- An Overview of Noise-Robust Automatic Speech RecognitionIEEE/ACM Transactions on Audio, Speech, and Language Processing, 2014
- Exploiting deep neural networks for detection-based speech recognitionNeurocomputing, 2013
- Using Broad Phonetic Group Experts for Improved Speech RecognitionIEEE Transactions on Audio, Speech, and Language Processing, 2007