Capturing Time Dynamics From Speech Using Neural Networks for Surgical Mask Detection
- 6 May 2022
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Journal of Biomedical and Health Informatics
- Vol. 26 (8) , 4291-4302
Abstract
The importance of detecting whether a per- son wears a face mask while speaking has tremendously increased since the outbreak of SARS-CoV-2 (COVID-19), as wearing a mask can help to reduce the spread of the virus and mitigate the public health crisis. Besides affecting human speech characteristics related to frequency, face masks cause temporal interferences in speech, altering the pace, rhythm, and pronunciation speed. In this regard, this paper presents two effective neural network models to detect surgical masks from audio. The proposed architectures are both based on Convolutional Neural Net- works (CNNs), chosen as an optimal approach for the spatial processing of the audio signals. One architecture applies a Long Short-Term Memory (LSTM) network to model the time-dependencies. Through an additional attention mechanism, the LSTM-based architecture enables the extraction of more salient temporal information. The other architecture (named ConvTx) retrieves the relative position of a sequence through the positional encoder of a transformer module. In order to assess to which extent both architectures can complement each other when modelling temporal dynamics, we also explore the combination of LSTM and Transformers in three hybrid models. Finally, we also investigate whether data augmentation techniques, such as, using transitions between audio frames and considering gender-dependent frameworks might impact the performance of the proposed architectures. Our experimental results show that one of the hybrid models achieves the best performance, surpassing existing state-of-the-art results for the task at hand.Keywords
Funding Information
- European Unions Horizon 2020 research the innovation programme under grant agreement sustAGE (No. 826506)
- BIT Teli Young Fellow Program
- China Scholarship Council (No. 202006680068)
This publication has 56 references indexed in Scilit:
- Effective data generation for imbalanced learning using conditional generative adversarial networksExpert Systems with Applications, 2018
- Self-Attention with Relative Position Representations2018
- CNN architectures for large-scale audio classification2017
- Deep Residual Learning for Image Recognition2016
- Data Augmentation for Deep Neural Network Acoustic ModelingIEEE/ACM Transactions on Audio, Speech, and Language Processing, 2015
- Deep learningNature, 2015
- Effective Approaches to Attention-based Neural Machine Translation2015
- The Boundaries of Language: Dealing with Paralinguistic FeaturesLingua Posnaniensis, 2012
- Receiver Operating Characteristic Curve in Diagnostic Test AssessmentJournal of Thoracic Oncology, 2010
- NOVELNEURALMODULATORSAnnual Review of Neuroscience, 2003