Block-Based High Performance CNN Architectures for Frame-Level Overlapping Speech Detection
- 6 November 2020
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE/ACM Transactions on Audio, Speech, and Language Processing
- Vol. 29 (23299290) , 28-40
Abstract
Speech technology systems such as Automatic Speech Recognition (ASR), speaker diarization, speaker recognition, and speech synthesis have advanced significantly by the emergence of deep learning techniques. However, none of these voice-enabled systems perform well in natural environmental circumstances, specifically in situations where one or more potential interfering talkers are involved. Therefore, overlapping speech detection has become an important front-end triage step for speech technology applications. This is crucial for large-scale datasets where manual labeling in not possible. A block-based CNN architecture is proposed to address modeling overlapping speech in audio streams with frames as short as 25 ms. The proposed architecture is robust to both: (i) shifts in distribution of network activations due to the change in network parameters during training, (ii) local variations from the input features caused by feature extraction, environmental noise, or room interference. We also investigate the effect of alternate input features including spectral magnitude, MFCC, MFB, and pyknogram on both computational time and classification performance. Evaluation is performed on simulated overlapping speech signals based on the GRID corpus. The experimental results highlight the capability of the proposed system in detecting overlapping speech frames with 90.5% accuracy, 93.5% precision, 92.7% recall, and 92.8% Fscore on same gender overlapped speech. For opposite gender cases, the network scores exceed 95% in all the classification metrics.Keywords
Funding Information
- National Science Foundation (# 1918032, # 2016725)
- University of Texas at Dallas
- Distinguished University Chair in Telecommunications Engineering
This publication has 25 references indexed in Scilit:
- Super-human multi-talker speech recognition: A graphical modeling approachComputer Speech & Language, 2010
- The cocktail party problemCurrent Biology, 2009
- The cocktail party problem: What is it? How can it be solved? And why should animal behaviorists study it?Journal of Comparative Psychology, 2008
- The Cocktail Party ProblemNeural Computation, 2005
- Nonlinear feature based classification of speech under stressIEEE Transactions on Speech and Audio Processing, 2001
- AM-FM energy detection and separation in noise using multiband energy operatorsIEEE Transactions on Signal Processing, 1993
- Energy separation in signal modulations with application to speech analysisIEEE Transactions on Signal Processing, 1993
- Attention: Some theoretical considerations.Psychological Review, 1963
- Contextual Cues in Selective ListeningQuarterly Journal of Experimental Psychology, 1960
- Some Experiments on the Recognition of Speech, with One and with Two EarsThe Journal of the Acoustical Society of America, 1953