Real Time Capture of Audio Images and their Use with Video
- 1 October 2007
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- No. 19311168,p. 10-13
Abstract
Spherical microphone arrays provide an ability to compute the acoustical intensity corresponding to different spatial directions in a given frame of audio-data. These intensities may be exhibited as an image and these images updated at a high frame rate to achieve a video stream if the data capture and intensity computations can be performed sufficiently quickly, there by creating a frame-rate audio camera. We describe how such a camera can be built and the processing done sufficiently quickly using graphics processors. The joint processing of captured frame-rate audio and video images enables applications such as visual identification of noise sources, beamforming and noise-suppression in video conferencing and others, provided it is possible to account for the spatial differences in the location of the audio and the video cameras. Based on the recognition that the spherical array can be viewed as a central projection camera it is possible to perform such joint analysis. We provide several examples of real-time applications.Keywords
This publication has 5 references indexed in Scilit:
- Flexible and Optimal Design of Spherical Microphone Arrays for BeamformingIEEE Transactions on Audio, Speech, and Language Processing, 2007
- Analysis and design of spherical microphone arraysIEEE Transactions on Speech and Audio Processing, 2004
- Plane-wave decomposition of the sound field on a sphere by spherical convolutionThe Journal of the Acoustical Society of America, 2004
- Spherical Microphone Arrays for 3D Sound Recording2004
- A detector tree of boosted classifiers for real-time object detection and tracking2003