Integrating Dilated Convolution into DenseLSTM for Audio Source Separation
Open Access
- 15 January 2021
- journal article
- research article
- Published by MDPI AG in Applied Sciences
- Vol. 11 (2) , 789
Abstract
Herein, we proposed a multi-scale multi-band dilated time-frequency densely connected convolutional network (DenseNet) with long short-term memory (LSTM) for audio source separation. Because the spectrogram of the acoustic signal can be thought of as images as well as time series data, it is suitable for convolutional recurrent neural network (CRNN) architecture. We improved the audio source separation performance by applying the dilated block with a dilated convolution to CRNN architecture. The dilated block has the role of effectively increasing the receptive field in the spectrogram. In addition, it was designed in consideration of the acoustic characteristics that the frequency axis and the time axis in the spectrogram are changed by independent influences such as speech rate and pitch. In speech enhancement experiments, we estimated the speech signal using various deep learning architectures from a signal in which the music, noise, and speech were mixed. We conducted the subjective evaluation on the estimated speech signal. In addition, speech quality, intelligibility, separation, and speech recognition performance were also measured. In music signal separation, we estimated the music signal using several deep learning architectures from the mixture of the music and speech signal. After that, the separation performance and music identification accuracy were measured using the estimated music signal. Overall, the proposed architecture shows the best performance compared to other deep learning architectures not only in speech experiments but also in music experiments.Keywords
This publication has 42 references indexed in Scilit:
- NMF-based Target Source Separation Using Deep Neural NetworkIEEE Signal Processing Letters, 2014
- From Blind to Guided Audio Source Separation: How models and side information can improve the separation of soundIEEE Signal Processing Magazine, 2014
- Objective measures for predicting speech intelligibility in noisy conditions based on new band-importance functionsThe Journal of the Acoustical Society of America, 2009
- Coherence and the speech intelligibility indexThe Journal of the Acoustical Society of America, 2005
- Gradient-based learning applied to document recognitionProceedings of the IEEE, 1998
- Long Short-Term MemoryNeural Computation, 1997
- Assessment for automatic speech recognition: II. NOISEX-92: A database and an experiment to study the effect of additive noise on speech recognition systemsSpeech Communication, 1993
- Backpropagation through time: what it does and how to do itProceedings of the IEEE, 1990
- Holographic implementation of a fully connected neural networkProceedings of the IEEE, 1990
- Receptive fields, binocular interaction and functional architecture in the cat's visual cortexThe Journal of Physiology, 1962