Exploiting Deep Neural Networks and Head Movements for Robust Binaural Localisation of Multiple Sources in Reverberant Environments
Authors:
- Ning MaNMNing Ma
- Tobias MayTMTobias May
- Guy J. BrownGBGuy J. Brown
Preprint
Abstract
This paper presents a novel machine-hearing system that exploits deep neural networks (DNNs) and head movements for robust binaural localisation of multiple sources in reverberant environments. DNNs are used to learn the relationship between the source azimuth and binaural cues, consisting of the complete cross-correlation function (CCF) and interaural level differences (ILDs). In contrast to many previous binaural hearing systems, the proposed approach is not restricted to localisation of sound sources in the frontal hemifield. Due to the similarity of binaural cues in the frontal and rear hemifields, front-back confusions often occur. To address this, a head movement strategy is incorporated in the localisation model to help reduce the front-back errors. The proposed DNN system is compared to a Gaussian mixture model (GMM) based system that employs interaural time differences (ITDs) and ILDs as localisation features. Our experiments show that the DNN is able to exploit information in the CCF that is not available in the ITD cue, which together with head movements substantially improves localisation accuracies under challenging acoustic scenarios in which multiple talkers and room reverberation are present.Keywords
All Related Versions
This publication has 17 references indexed in Scilit:
- Binaural Classification for Reverberant Speech Segregation Using Deep Neural NetworksIEEE/ACM Transactions on Audio, Speech, and Language Processing, 2014
- Localization of Multiple Speakers under High Reverberation using a Spherical Microphone Array and the Direct-Path Dominance TestIEEE/ACM Transactions on Audio, Speech, and Language Processing, 2014
- Binaural Localization of Multiple Sources in Reverberant and Noisy EnvironmentsIEEE Transactions on Audio, Speech, and Language Processing, 2012
- Dynamic Precedence Effect Modeling for Source Separation in Reverberant EnvironmentsIEEE Transactions on Audio, Speech, and Language Processing, 2010
- A Probabilistic Model for Robust Localization Based on a Binaural Auditory Front-EndIEEE Transactions on Audio, Speech, and Language Processing, 2010
- An audio-visual corpus for speech perception and automatic speech recognitionThe Journal of the Acoustical Society of America, 2006
- A Probabilistic Model for Binaural Sound LocalizationIEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), 2006
- Source localization in complex listening situations: Selection of binaural cues based on interaural coherenceThe Journal of the Acoustical Society of America, 2004
- Role of spectral cues in median plane localizationThe Journal of the Acoustical Society of America, 1990
- The role of head movements and vestibular and visual cues in sound localization.Journal of Experimental Psychology, 1940