Multisensory Integration Sites Identified by Perception of Spatial Wavelet Filtered Visual Speech Gesture Information
- 1 June 2004
- journal article
- research article
- Published by MIT Press in Journal of Cognitive Neuroscience
- Vol. 16 (5) , 805-816
Abstract
Perception of speech is improved when presentation of the audio signal is accompanied by concordant visual speech gesture information. This enhancement is most prevalent when the audio signal is degraded. One potential means by which the brain affords perceptual enhancement is thought to be through the integration of concordant information from multiple sensory channels in a common site of convergence, multisensory integration (MSI) sites. Some studies have identified potential sites in the superior temporal gyrus/sulcus (STG/S) that are responsive to multisensory information from the auditory speech signal and visual speech movement. One limitation of these studies is that they do not control for activity resulting from attentional modulation cued by such things as visual information signaling the onsets and offsets of the acoustic speech signal, as well as activity resulting from MSI of properties of the auditory speech signal with aspects of gross visual motion that are not specific to place of articulation information. This fMRI experiment uses spatial wavelet bandpass filtered Japanese sentences presented with background multispeaker audio noise to discern brain activity reflecting MSI induced by auditory and visual correspondence of place of articulation information that controls for activity resulting from the above-mentioned factors. The experiment consists of a low-frequency (LF) filtered condition containing gross visual motion of the lips, jaw, and head without specific place of articulation information, a midfrequency (MF) filtered condition containing place of articulation information, and an unfiltered (UF) condition. Sites of MSI selectively induced by auditory and visual correspondence of place of articulation information were determined by the presence of activity for both the MF and UF conditions relative to the LF condition. Based on these criteria, sites of MSI were found predominantly in the left middle temporal gyrus (MTG), and the left STG/S (including the auditory cortex). By controlling for additional factors that could also induce greater activity resulting from visual motion information, this study identifies potential MSI sites that we believe are involved with improved speech perception intelligibility.Keywords
This publication has 50 references indexed in Scilit:
- A Functional-Anatomical Model for LipreadingJournal of Neurophysiology, 2003
- Brain activity during audiovisual speech perception: An fMRI study of the McGurk effectNeuroReport, 2003
- Thresholding of Statistical Maps in Functional Neuroimaging Using the False Discovery RateNeuroImage, 2002
- Detection of Audio-Visual Integration Sites in Humans by Application of Electrophysiological Criteria to the BOLD EffectNeuroImage, 2001
- Social perception from visual cues: role of the STS regionTrends in Cognitive Sciences, 2000
- The functional anatomy of visual-tactile integration in man: a study using positron emission tomographyNeuropsychologia, 2000
- Language within our graspTrends in Neurosciences, 1998
- A new brain region for coordinating speech articulationNature, 1996
- Evaluating the articulation index for auditory–visual inputThe Journal of the Acoustical Society of America, 1991
- Hearing lips and seeing voicesNature, 1976