A Four-Stage Data Augmentation Approach to ResNet-Conformer Based Acoustic Modeling for Sound Event Localization and Detection
Top Cited Papers
- 13 March 2023
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE/ACM Transactions on Audio, Speech, and Language Processing
- Vol. 31 (23299290) , 1251-1264
Abstract
In this paper, we propose a novel four-stage data augmentation approach to ResNet-Conformer based acoustic modeling for sound event localization and detection (SELD). First, we explore two spatial augmentation techniques, namely audio channel swapping (ACS) and multi-channel simulation (MCS), to deal with data sparsity in SELD. ACS and MDS focus on augmenting the limited training data with expanding direction of arrival (DOA) representations such that the acoustic models trained with the augmented data are robust to localization variations of acoustic sources. Next, time-domain mixing (TDM) and time-frequency masking (TFM) are also investigated to deal with overlapping sound events and data diversity. Finally, ACS, MCS, TDM and TFM are combined in a step-by-step manner to form an effective four-stage data augmentation scheme. Tested on the Detection and Classification of Acoustic Scenes and Events (DCASE) 2020 data set, our proposed augmentation approach greatly improves the system performance, ranking our submitted system in the first place in the SELD task of the DCASE 2020 Challenge. Furthermore, we employ a ResNet-Conformer architecture to model both global and local context dependencies of an audio sequence and win the first place in the DCASE 2022 SELD evaluations.Keywords
Funding Information
- National Natural Science Foundation of China (62171427)
This publication has 59 references indexed in Scilit:
- Data Augmentation for Deep Neural Network Acoustic ModelingIEEE/ACM Transactions on Audio, Speech, and Language Processing, 2015
- Robust sound event recognition using convolutional neural networks2015
- Sound event detection in real life recordings using coupled matrix factorization of spectral representations and class activity annotations2015
- Robust Sound Event Classification Using Deep Neural NetworksIEEE/ACM Transactions on Audio, Speech, and Language Processing, 2015
- Convolutional Neural Networks for Distant Speech RecognitionIEEE Signal Processing Letters, 2014
- Context-dependent sound event detectionEURASIP Journal on Audio, Speech, and Music Processing, 2013
- Acoustic source detection and localization based on wavefield decomposition using circular microphone arraysThe Journal of the Acoustical Society of America, 2006
- ESPRIT-estimation of signal parameters via rotational invariance techniquesIEEE Transactions on Acoustics, Speech, and Signal Processing, 1989
- Multiple emitter location and signal parameter estimationIEEE Transactions on Antennas and Propagation, 1986
- The generalized correlation method for estimation of time delayIEEE Transactions on Acoustics, Speech, and Signal Processing, 1976