Sound Event Detection in Domestic Environment Using Frequency-Dynamic Convolution and Local Attention
Open Access
- 30 September 2023
- journal article
- research article
- Published by MDPI AG in Information
- Vol. 14 (10) , 534
Abstract
This work describes a methodology for sound event detection in domestic environments. Efficient solutions in this task can support the autonomous living of the elderly. The methodology deals with the “Challenge on Detection and Classification of Acoustic Scenes and Events (DCASE)” 2023, and more specifically with Task 4a “Sound event detection of domestic activities”. This task involves the detection of 10 common events in domestic environments in 10 s sound clips. The events may have arbitrary duration in the 10 s clip. The main components of the methodology are data augmentation on mel-spectrograms that represent the sound clips, feature extraction by passing spectrograms through a frequency-dynamic convolution network with an extra attention module in sequence with each convolution, concatenation of these features with BEATs embeddings, and use of BiGRU for sequence modeling. Also, a mean teacher model is employed for leveraging unlabeled data. This research focuses on the effect of data augmentation techniques, of the feature extraction models, and on self-supervised learning. The main contribution is the proposed feature extraction model, which uses weighted attention on frequency in each convolution, combined in sequence with a local attention module adopted by computer vision. The proposed system features promising and robust performance.Keywords
Funding Information
- Improvement of the Quality of Life and Activity for the Elderly (MIS 5047294, NSRF 2014-2020)
- Competitiveness, Entrepreneurship and Innovation (MIS 5047294, NSRF 2014-2020)
- European Regional Development Fund (MIS 5047294, NSRF 2014-2020)
This publication has 22 references indexed in Scilit:
- HTS-AT: A Hierarchical Token-Semantic Audio Transformer for Sound Classification and Detection2022
- Multi-Scale Features for Transformer Model to Improve the Performance of Sound Event DetectionApplied Sciences, 2022
- PSLA: Improving Audio Tagging With Pretraining, Sampling, Labeling, and AggregationIEEE/ACM Transactions on Audio, Speech, and Language Processing, 2021
- BYOL for Audio: Self-Supervised Learning for General-Purpose Audio Representation2021
- Sound Event Detection by Consistency Training and Pseudo-Labeling With Feature-Pyramid Convolutional Recurrent Neural Networks2021
- A Framework for the Robust Evaluation of Sound Event Detection2020
- Sound Event Detection in Synthetic Domestic Environments2020
- Audio Set: An ontology and human-labeled dataset for audio events2017
- Monitoring Activities of Daily Living in Smart Homes: Understanding human behaviorIEEE Signal Processing Magazine, 2016
- SVM-Based Multimodal Classification of Activities of Daily Living in Health Smart Homes: Sensors, Algorithms, and First Experimental ResultsIEEE Transactions on Information Technology in Biomedicine, 2009