Multichannel Sound Event Detection Using 3D Convolutional Neural Networks for Learning Inter-channel Features
- 1 July 2018
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 1-7
Abstract
In this paper, we propose a stacked convolutional and recurrent neural network (CRNN) with a 3D convolutional neural network (CNN) in the first layer for the multichannel sound event detection (SED) task. The 3D CNN enables the network to simultaneously learn the inter-and intra-channel features from the input multichannel audio. In order to evaluate the proposed method, multichannel audio datasets with different number of overlapping sound sources are synthesized. Each of this dataset has a four-channel first-order Ambisonic, binaural, and single-channel versions, on which the performance of SED using the proposed method are compared to study the potential of SED using multichannel audio. A similar study is also done with the binaural and single-channel versions of the real-life recording TUT-SED 2017 development dataset. The proposed method learns to recognize overlapping sound events from multichannel features faster and performs better SED with a fewer number of training epochs. The results show that on using multichannel Ambisonic audio in place of single-channel audio we improve the overall F-score by 7.5%, overall error rate by 10% and recognize 15.6% more sound events in time frames with four overlapping sound sources.Keywords
All Related Versions
This publication has 14 references indexed in Scilit:
- Sound event detection using spatial features and convolutional recurrent neural network2017
- Robust Audio Event Recognition with 1-Max Pooling Convolutional Neural Networks2016
- Metrics for Polyphonic Sound Event DetectionApplied Sciences, 2016
- Audio SurveillanceACM Computing Surveys, 2016
- Learning Spatiotemporal Features with 3D Convolutional Networks2015
- Robust sound event recognition using convolutional neural networks2015
- Estimating animal population density using passive acousticsBiological Reviews, 2012
- Environmental Sound Recognition With Time–Frequency Audio FeaturesIEEE Transactions on Audio, Speech, and Language Processing, 2009
- Acoustic Event Detection: SVM-Based System and Evaluation Setup in CLEAR’07Lecture Notes in Computer Science, 2007
- The generalized correlation method for estimation of time delayIEEE Transactions on Acoustics, Speech, and Signal Processing, 1976