Seeing Emotion with Your Ears: Emotional Prosody Implicitly Guides Visual Attention to Faces
Open Access
- 30 January 2012
- journal article
- research article
- Published by Public Library of Science (PLoS) in PLOS ONE
- Vol. 7 (1) , e30740
Abstract
Interpersonal communication involves the processing of multimodal emotional cues, particularly facial expressions (visual modality) and emotional speech prosody (auditory modality) which can interact during information processing. Here, we investigated whether the implicit processing of emotional prosody systematically influences gaze behavior to facial expressions of emotion. We analyzed the eye movements of 31 participants as they scanned a visual array of four emotional faces portraying fear, anger, happiness, and neutrality, while listening to an emotionally-inflected pseudo-utterance (Someone migged the pazing) uttered in a congruent or incongruent tone. Participants heard the emotional utterance during the first 1250 milliseconds of a five-second visual array and then performed an immediate recall decision about the face they had just seen. The frequency and duration of first saccades and of total looks in three temporal windows ([0–1250 ms], [1250–2500 ms], [2500–5000 ms]) were analyzed according to the emotional content of faces and voices. Results showed that participants looked longer and more frequently at faces that matched the prosody in all three time windows (emotion congruency effect), although this effect was often emotion-specific (with greatest effects for fear). Effects of prosody on visual attention to faces persisted over time and could be detected long after the auditory information was no longer present. These data imply that emotional prosody is processed automatically during communication and that these cues play a critical role in how humans respond to related visual cues in the environment, such as facial expressions.Keywords
This publication has 104 references indexed in Scilit:
- Neural correlates of cross-domain affective primingBrain Research, 2010
- Remembering the Details: Effects of EmotionEmotion Review, 2009
- Neural processing of vocal emotion and identityBrain and Cognition, 2009
- Seeing who we hear and hearing who we seeProceedings of the National Academy of Sciences, 2009
- On the give and take between event apprehension and utterance formulationJournal of Memory and Language, 2007
- The N2pc component and its links to attention shifts and spatially selective visual processingPsychophysiology, 2007
- Retrieval of emotional memories.Psychological Bulletin, 2007
- The eyes are sufficient to produce a threat superiority effect.Emotion, 2006
- Facial Expressions of Emotion: Are Angry Faces Detected More Efficiently?Cognition and Emotion, 2000
- Hearing lips and seeing voicesNature, 1976