U-Shaped Transformer With Frequency-Band Aware Attention for Speech Enhancement
- 12 April 2023
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE/ACM Transactions on Audio, Speech, and Language Processing
- Vol. 31 (23299290) , 1511-1521
Abstract
Recently, Transformer shows the potential to exploit the long-range sequence dependency in speech with self-attention. It has been introduced in single channel speech enhancement to improve the accuracy of speech estimation from a noise mixture. However, the amount of information represented across attention-heads is often huge, which leads to increased computational complexity. To address this issue, the axial attention is proposed i.e., to split a 2D attention into two 1-D attentions. In this paper, we develop a new method for speech enhancement by leveraging the axial attention, where we generate time and frequency sub-attention maps by calculating the attention map along time- and frequency-axis. Different from the conventional axial attention, the proposed method provides two parallel multi-head attentions for time- and frequency-axis, respectively. Moreover, the frequency-band aware attention is proposed i.e., high frequency-band attention (HFA), and low frequency-band attention (LFA), which facilitates the exploitation of the information related to speech and noise in different frequency bands in the noisy mixture. To re-use high-resolution feature maps from the encoder, we design a U-shaped Transformer, which helps recover lost information from the high-level representations to further improve the speech estimation accuracy. Extensive experiments on four public datasets are used to demonstrate the efficacy of the proposed method. The code of the proposed method is available at https://github.com/Yukino-3/U-shaped-Transformer-SE .Keywords
This publication has 51 references indexed in Scilit:
- Supervised Speech Separation Based on Deep Learning: An OverviewIEEE/ACM Transactions on Audio, Speech, and Language Processing, 2018
- A new data-driven band-weighting function for predicting the intelligibility of noise-suppressed speech2017
- A Fully Convolutional Neural Network for Speech Enhancement2017
- Deep Residual Learning for Image Recognition2016
- U-Net: Convolutional Networks for Biomedical Image Segmentation2015
- On Training Targets for Supervised Speech SeparationIEEE/ACM Transactions on Audio, Speech, and Language Processing, 2014
- The voice bank corpus: Design, collection and data analysis of a large regional accent speech database2013
- The diverse environments multi-channel acoustic noise database: A database of multichannel environmental noise recordingsThe Journal of the Acoustical Society of America, 2013
- Depth-kymography: high-speed calibrated 3D imaging of human vocal fold vibration dynamicsPhysics in Medicine & Biology, 2008
- Evaluation of Objective Quality Measures for Speech EnhancementIEEE Transactions on Audio, Speech, and Language Processing, 2007