Dual-Branch Attention-In-Attention Transformer for Single-Channel Speech Enhancement
Authors:
- 23 May 2022
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 7847-7851
Abstract
Curriculum learning begins to thrive in the speech enhancement area, which decouples the original spectrum estimation task into multiple easier sub-tasks to achieve better performance. Motivated by that, we propose a dual-branch attention-in-attention transformer dubbed DB-AIAT to handle both coarse- and fine-grained regions of the spectrum in parallel. From a complementary perspective, a magnitude masking branch is proposed to coarsely estimate the overall magnitude spectrum, and simultaneously a complex refining branch is elaborately designed to compensate for the missing spectral details and implicitly derive phase information. Within each branch, we propose a novel attention-in-attention transformer-based module to replace the conventional RNNs and temporal convolutional networks for temporal sequence modeling. Specifically, the proposed attention-in-attention transformer consists of adaptive temporal-frequency attention transformer blocks and an adaptive hierarchical attention module, aiming to capture long-term temporal-frequency dependencies and further aggregate global hierarchical contextual information. Experimental results on Voice Bank + DEMAND demonstrate that DB-AIAT yields state-of-the-art performance (e.g., 3.31 PESQ, 95.6% STOI and 10.79dB SSNR) over previous advanced systems with a relatively small model size (2.81M).Keywords
All Related Versions
Funding Information
- National Natural Science Foundation of China
This publication has 27 references indexed in Scilit:
- Supervised Speech Separation Based on Deep Learning: An OverviewIEEE/ACM Transactions on Audio, Speech, and Language Processing, 2018
- Time-Frequency Masking-Based Speech Enhancement Using Generative Adversarial Network2018
- A Regression Approach to Speech Enhancement Based on Deep Neural NetworksIEEE/ACM Transactions on Audio, Speech, and Language Processing, 2014
- On Training Targets for Supervised Speech SeparationIEEE/ACM Transactions on Audio, Speech, and Language Processing, 2014
- The voice bank corpus: Design, collection and data analysis of a large regional accent speech database2013
- The diverse environments multi-channel acoustic noise database: A database of multichannel environmental noise recordingsThe Journal of the Acoustical Society of America, 2013
- Speech Enhancement2013
- The importance of phase in speech enhancementSpeech Communication, 2011
- Evaluation of Objective Quality Measures for Speech EnhancementIEEE Transactions on Audio, Speech, and Language Processing, 2007
- Perceptual evaluation of speech quality (PESQ)-a new method for speech quality assessment of telephone networks and codecs2002