Cross-Domain Conv-TasNet Speech Enhancement Model with Two-Level Bi-Projection Fusion of Discrete Wavelet Transform
Open Access
- 12 May 2023
- journal article
- research article
- Published by MDPI AG in Applied Sciences
- Vol. 13 (10) , 5992
Abstract
Nowadays, time-domain features see wide use in speech enhancement (SE) networks such as frequency-domain features to achieve excellent performance in eliminating noise from input utterances. This study primarily investigates how to extract information from time-domain utterances to create more effective features in SE. We extend our recent work by employing sub-signals which dwell in multiple acoustic frequency bands in the time domain and integrating them into a unified time-domain feature set. The discrete wavelet transform (DWT) is applied to decompose each input frame signal to obtain sub-band signals, and a projection fusion process is performed on these signals to create the ultimate features. The corresponding fusion strategy is either bi-projection fusion (BPF) or multiple projection fusion (MPF). In short, MPF exploits the softmax function to replace the sigmoid function in order to create ratio masks for multiple feature sources. The concatenation of fused DWT features and time features serves as the encoder output of two celebrated SE frameworks, the fully convolutional time-domain audio separation network (Conv-TasNet) and the dual-path transformer network (DPTNet), to estimate the mask and then produce the enhanced time-domain utterances. The evaluation experiments are conducted on the VoiceBank-DEMAND and VoiceBank-QUT tasks, and the results reveal that the proposed method achieves higher speech quality and intelligibility than the original Conv-TasNet that uses time features only, indicating that the fusion of DWT features created from the input utterances can benefit time features to learn a superior Conv-TasNet/DPTNet network in SE.Keywords
Funding Information
- National Science and Technology Council (111-2221-E-260-005-MY2)
This publication has 40 references indexed in Scilit:
- Phase-sensitive and recognition-boosted speech separation using deep recurrent neural networks2015
- A Regression Approach to Speech Enhancement Based on Deep Neural NetworksIEEE/ACM Transactions on Audio, Speech, and Language Processing, 2014
- On Training Targets for Supervised Speech SeparationIEEE/ACM Transactions on Audio, Speech, and Language Processing, 2014
- The voice bank corpus: Design, collection and data analysis of a large regional accent speech database2013
- Ideal ratio mask estimation using deep neural networks for robust speech recognition2013
- Maximum likelihood linear transformations for HMM-based speech recognitionComputer Speech & Language, 1998
- Maximum likelihood linear regression for speaker adaptation of continuous density hidden Markov modelsComputer Speech & Language, 1995
- Maximum a posteriori estimation for multivariate Gaussian mixture observations of Markov chainsIEEE Transactions on Speech and Audio Processing, 1994
- Speech enhancement using a minimum mean-square error log-spectral amplitude estimatorIEEE Transactions on Acoustics, Speech, and Signal Processing, 1985
- Suppression of acoustic noise in speech using spectral subtractionIEEE Transactions on Acoustics, Speech, and Signal Processing, 1979