Hearing and seeing meaning in noise: Alpha, beta, and gamma oscillations predict gestural enhancement of degraded speech comprehension
Open Access
- 30 January 2018
- journal article
- research article
- Published by Wiley in Human Brain Mapping
- Vol. 39 (5) , 2075-2087
Abstract
During face‐to‐face communication, listeners integrate speech with gestures. The semantic information conveyed by iconic gestures (e.g., a drinking gesture) can aid speech comprehension in adverse listening conditions. In this magnetoencephalography (MEG) study, we investigated the spatiotemporal neural oscillatory activity associated with gestural enhancement of degraded speech comprehension. Participants watched videos of an actress uttering clear or degraded speech, accompanied by a gesture or not and completed a cued‐recall task after watching every video. When gestures semantically disambiguated degraded speech comprehension, an alpha and beta power suppression and a gamma power increase revealed engagement and active processing in the hand‐area of the motor cortex, the extended language network (LIFG/pSTS/STG/MTG), medial temporal lobe, and occipital regions. These observed low‐ and high‐frequency oscillatory modulations in these areas support general unification, integration and lexical access processes during online language comprehension, and simulation of and increased visual attention to manual gestures over time. All individual oscillatory power modulations associated with gestural enhancement of degraded speech comprehension predicted a listener's correct disambiguation of the degraded verb after watching the videos. Our results thus go beyond the previously proposed role of oscillatory dynamics in unimodal degraded speech comprehension and provide first evidence for the role of low‐ and high‐frequency oscillations in predicting the integration of auditory and visual information at a semantic level.Keywords
Funding Information
- Netherlands Organisation for Scientific Research (024.001.006)
This publication has 68 references indexed in Scilit:
- Vividness of Visual Imagery and Incidental Recall of Verbal Cues, When Phenomenological Availability Reflects Long-Term Memory AccessibilityFrontiers in Psychology, 2013
- A Supramodal Neural Network for Speech and Gesture Semantics: An fMRI StudyPLOS ONE, 2012
- “Too Many betas do not Spoil the Broth”: The Role of Beta Brain Oscillations in Language ProcessingFrontiers in Psychology, 2012
- Integration or Predictability? A Further Specification of the Functional Role of Gamma Oscillations in Language ComprehensionFrontiers in Psychology, 2012
- Alpha Rhythms in Audition: Cognitive and Clinical PerspectivesFrontiers in Psychology, 2011
- FieldTrip: Open Source Software for Advanced Analysis of MEG, EEG, and Invasive Electrophysiological DataComputational Intelligence and Neuroscience, 2010
- Shaping Functional Architecture by Oscillatory Alpha Activity: Gating by InhibitionFrontiers in Human Neuroscience, 2010
- Relationship between Activity in Human Primary Motor Cortex during Action Observation and the Mirror Neuron SystemPLOS ONE, 2009
- At the heart of the ventral attention system: The right anterior insulaHuman Brain Mapping, 2008
- Actor's and observer's primary motor cortices stabilize similarly after seen or heard motor actionsProceedings of the National Academy of Sciences, 2007