Exploiting Deep Neural Networks and Head Movements for Robust Binaural Localization of Multiple Sources in Reverberant Environments
Open Access
- 27 October 2017
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE/ACM Transactions on Audio, Speech, and Language Processing
- Vol. 25 (12) , 2444-2453
Abstract
This paper presents a novel machine-hearing system that exploits deep neural networks (DNNs) and head movements for robust binaural localization of multiple sources in reverberant environments. DNNs are used to learn the relationship between the source azimuth and binaural cues, consisting of the complete cross-correlation function (CCF) and interaural level differences (ILDs). In contrast to many previous binaural hearing systems, the proposed approach is not restricted to localization of sound sources in the frontal hemifield. Due to the similarity of binaural cues in the frontal and rear hemifields, front-back confusions often occur. To address this, a head movement strategy is incorporated in the localization model to help reduce the front-back errors. The proposed DNN system is compared to a Gaussian-mixture-model-based system that employs interaural time differences (ITDs) and ILDs as localization features. Our experiments show that the DNN is able to exploit information in the CCF that is not available in the ITD cue, which together with head movements substantially improves localization accuracies under challenging acoustic scenarios, in which multiple talkers and room reverberation are present.Keywords
All Related Versions
Funding Information
- European Union FP7 Project (618075)
This publication has 22 references indexed in Scilit:
- Binaural Classification for Reverberant Speech Segregation Using Deep Neural NetworksIEEE/ACM Transactions on Audio, Speech, and Language Processing, 2014
- Sound localization with head movement: implications for 3-d audio displaysFrontiers in Neuroscience, 2014
- Localization of Multiple Speakers under High Reverberation using a Spherical Microphone Array and the Direct-Path Dominance TestIEEE/ACM Transactions on Audio, Speech, and Language Processing, 2014
- A Binaural Model that Analyses Acoustic Spaces and Stereophonic Reproduction Systems by Utilizing Head Rotations2013
- Binaural Localization and Detection of Speakers in Complex Acoustic Scenes2013
- Learning Deep Architectures for AIFoundations and Trends in Machine Learning, 2009
- The effect of head rotations on vertical plane sound localizationThe Journal of the Acoustical Society of America, 1997
- A computational theory of spectral cue localizationThe Journal of the Acoustical Society of America, 1993
- Role of spectral cues in median plane localizationThe Journal of the Acoustical Society of America, 1990
- The role of head movements and vestibular and visual cues in sound localization.Journal of Experimental Psychology, 1940