Hillel Pratt

Introduction: Spatio‐temporal distributions of cortical activity to audio‐visual presentations of meaningless vowel‐consonant‐vowels and the effects of audio‐visual congruence/incongruence, with emphasis on the McGurk effect, were studied. The McGurk effect occurs when a clearly audible syllable with one consonant, is presented simultaneously with a visual presentation of a face articulating a syllable with a different consonant and the resulting percept is a syllable with a consonant other than the auditorily presented one.Methods: Twenty subjects listened to pairs of audio‐visually congruent or incongruent utterances and indicated whether pair members were the same or not. Source current densities of event‐related potentials to the first utterance in the pair were estimated and effects of stimulus–response combinations, brain area, hemisphere, and clarity of visual articulation were assessed.Results: Auditory cortex, superior parietal cortex, and middle temporal cortex were the most consistently involved areas across experimental conditions. Early (<200 msec) processing of the consonant was overall prominent in the left hemisphere, except right hemisphere prominence in superior parietal cortex and secondary visual cortex. Clarity of visual articulation impacted activity in secondary visual cortex and Wernicke's area. McGurk perception was associated with decreased activity in primary and secondary auditory cortices and Wernicke's area before 100 msec, increased activity around 100 msec which decreased again around 180 msec. Activity in Broca's area was unaffected by McGurk perception and was only increased to congruent audio‐visual stimuli 30–70 msec following consonant onset.Conclusions: The results suggest left hemisphere prominence in the effects of stimulus and response conditions on eight brain areas involved in dynamically distributed parallel processing of audio‐visual integration. Initially (30–70 msec) subcortical contributions to auditory cortex, superior parietal cortex, and middle temporal cortex occur. During 100–140 msec, peristriate visual influences and Wernicke's area join in the processing. Resolution of incongruent audio‐visual inputs is then attempted, and if successful, McGurk perception occurs and cortical activity in left hemisphere further increases between 170 and 260 msec.

The study determined how spatiotemporal distribution of cortical activity to words in first and second language is affected by language, proficiency, and linguistic setting. Ten early bilinguals and 14 late adult bilinguals listened to pairs of words presented in Arabic (L1), Hebrew (L2), or in mixed pairs and indicated whether both words had the same meaning or not. Source current densities of event‐related potentials were estimated. Activity to first words in the pair lateralized to right hemisphere, higher to L1 than L2 during early processing (300 ms). During early and late processing, activities were larger in mixed than monolinguistic settings among early bilinguals but lower in mixed than in monolinguistic settings among late bilinguals. Late processing in auditory regions was of larger magnitude in left than right hemispheres among both groups. Activity to second words in the pair was larger in mixed than in monolinguistic settings during both early and late processing among both groups. Early processing of second words in auditory regions lateralized to the right among early bilinguals and to the left among late bilinguals, whereas late processing did not differ between groups. Wernicke's area activity during late processing of L2 was larger on the right, while on the left no significant differences between languages were found. The results show that cortical language processing in bilinguals differs between early and late processing and these differences are modulated by linguistic proficiency and setting. Hum Brain Mapp 34:2863–2881, 2013.

This study determined the effects of phonology and semantics on the distribution of cortical activity to the second of a pair of words in first and second language (mixed pairs). The effects of relative proficiency in the two languages and linguistic setting (monolinguistic or mixed) are reported in a companion paper. Ten early bilinguals and 14 late bilinguals listened to mixed pairs of words in Arabic (L1) and Hebrew (L2) and indicated whether both words in the pair had the same or different meanings. The spatio‐temporal distribution of current densities of event‐related potentials were estimated for each language and according to semantic and phonologic relationship (same or different) compared with the first word in the pair. During early processing (300 ms) were enhanced by semantic incongruence between the two words, particularly in temporal areas and in left hemisphere Broca's and Wernicke's areas. The latter differences were greater when words were in L2. Surprisingly, no significant effects of relative proficiency on processing the second word in the pair were found. These results indicate that the distribution of brain activity to the second of two words presented bilingually is affected differently during early and late processing by both semantic and phonologic priming by‐ and incongruence with the immediately preceding word. Hum Brain Mapp 34:2882–2898, 2013.