Multichannel Speech Enhancement Based on Neural Beamforming and a Context-Focused Post-Filtering Network
- 15 September 2023
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Cognitive and Developmental Systems
- Vol. 16 (3) , 973-983
Abstract
Both spatial and temporal contextual information are essential for the multichannel speech enhancement (MCSE) task. In this work, we propose a unified MCSE network composed of neural beamforming and a context-focused post-filtering network in order to fully exploit these two types of information. The network is used to estimate the optimum complex ideal ratio masks (cIRM) which can more effectively utilize the phase information in the frequency domain to reconstruct the speech waveform. To assign adaptive weights to channels, we first adopt a dilated convolution-based network which simulates the beamforming on the original multichannel input spectrum as the front-end of the multichannel acoustic model. Furthermore, we propose a post-filtering network that inputs the suggested U-Net’s output to a convolutional long short-term memory (ConvLSTM) layer, as it can properly capture contextual information and spatial correlation information of features. We conduct experiments on the VOICES, CHiME-3 and WMIR datasets, respectively. Experiments show that, in various scenarios, the proposed algorithm shows improvements over the previous state-of-the-art algorithms in terms of PESQ, STOI and SI-SNR.Keywords
Funding Information
- National Key Research and Development Program of China (2020YFC2004002, 2020YFC2004003)
- National Natural Science Foundation of China (62001215)
- Key Research and Development Program of Jiangsu Province (BE2022059-3)
This publication has 54 references indexed in Scilit:
- A Review of the State of the Art and Future Challenges of Deep Learning-Based BeamformingIEEE Access, 2022
- An End-to-End Far-Field Keyword Spotting System with Neural Beamforming2021
- Speech emotion recognition using convolutional long short-term memory neural network and support vector machines2017
- Design of multichannel wiener filter for speech enhancement in hearing aids and noise reduction technique2016
- Neural network based spectral mask estimation for acoustic beamforming2016
- Deep beamforming networks for multi-channel speech recognition2016
- The third ‘CHiME’ speech separation and recognition challenge: Dataset, task and baselines2015
- U-Net: Convolutional Networks for Biomedical Image Segmentation2015
- Perceptual evaluation of speech quality (PESQ)-a new method for speech quality assessment of telephone networks and codecs2002
- Assessment for automatic speech recognition: II. NOISEX-92: A database and an experiment to study the effect of additive noise on speech recognition systemsSpeech Communication, 1993