Audiovisual integration as conflict resolution: The conflict of the McGurk illusion
- 9 August 2017
- journal article
- research article
- Published by Wiley in Human Brain Mapping
- Vol. 38 (11) , 5691-5705
Abstract
There are two main behavioral expressions of multisensory integration (MSI) in speech; the perceptual enhancement produced by the sight of the congruent lip movements of the speaker, and the illusory sound perceived when a speech syllable is dubbed with incongruent lip movements, in the McGurk effect. These two models have been used very often to study MSI. Here, we contend that, unlike congruent audiovisually (AV) speech, the McGurk effect involves brain areas related to conflict detection and resolution. To test this hypothesis, we used fMRI to measure blood oxygen level dependent responses to AV speech syllables. We analyzed brain activity as a function of the nature of the stimuli—McGurk or non‐McGurk—and the perceptual outcome regarding MSI—integrated or not integrated response—in a 2 × 2 factorial design. The results showed that, regardless of perceptual outcome, AV mismatch activated general‐purpose conflict areas (e.g., anterior cingulate cortex) as well as specific AV speech conflict areas (e.g., inferior frontal gyrus), compared with AV matching stimuli. Moreover, these conflict areas showed stronger activation on trials where the McGurk illusion was perceived compared with non‐illusory trials, despite the stimuli where physically identical. We conclude that the AV incongruence in McGurk stimuli triggers the activation of conflict processing areas and that the process of resolving the cross‐modal conflict is critical for the McGurk illusion to arise. Hum Brain Mapp 38:5691–5705, 2017. © 2017 Wiley Periodicals, Inc.Keywords
Funding Information
- Ministerio de Economía y Competitividad (PSI2016‐75558‐PAEI/FEDER)
- European Research Council (StG‐2010 263145)
This publication has 69 references indexed in Scilit:
- A neural basis for interindividual differences in the McGurk effect, a multisensory speech illusionNeuroImage, 2012
- Discrete neural substrates underlie complementary audiovisual speech integration processesNeuroImage, 2011
- Psychophysics of the McGurk and other audiovisual speech integration effects.Journal of Experimental Psychology: Human Perception and Performance, 2011
- Multisensory conflict modulates the spread of visual attention across a multisensory objectNeuroImage, 2010
- Primary and multisensory cortical activity is correlated with audiovisual perceptsHuman Brain Mapping, 2010
- Neural processing of asynchronous audiovisual speech perceptionNeuroImage, 2010
- Quantified acoustic–optical speech signal incongruity identifies cortical sites of audiovisual speech processingBrain Research, 2008
- Spatiotemporal dynamics of audiovisual speech processingNeuroImage, 2008
- Abstract Coding of Audiovisual Speech: Beyond Sensory RepresentationNeuron, 2007
- Hearing lips and seeing voicesNature, 1976