The Effect of Audio-Visual Smiles on Social Influence in a Cooperative Human–Agent Interaction Task
Authors:
Open Access
- 15 November 2021
- journal article
- research article
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Computer-Human Interaction
- Vol. 28 (6) , 1-38
Abstract
Emotional expressivity is essential for human interactions, informing both perception and decision-making. Here, we examine whether creating an audio-visual emotional channel mismatch influences decision-making in a cooperative task with a virtual character. We created a virtual character that was either congruent in its emotional expression (smiling in the face and voice) or incongruent (smiling in only one channel). People (N = 98) evaluated the character in terms of valence and arousal in an online study; then, visitors in a museum played the “lunar survival task” with the character over three experiments (N = 597, 78, 101, respectively). Exploratory results suggest that multi-modal expressions are perceived, and reacted upon, differently than unimodal expressions, supporting previous theories of audio-visual integration.Keywords
Funding Information
- European Union’s Horizon 2020 Research and Innovation Programme
- Marie Skłodowska-Curie (713567)
- ADAPT Centre for Digital Content Technology
- SFI Research Centres Programme (13/RC/2016)
- European Regional Development Fund
- Science Foundation Ireland, Game Face (13/CDA/2135)
This publication has 101 references indexed in Scilit:
- Physical attractiveness influences reproductive success of modern menJournal of Ethology, 2011
- The Mystery of the European Smile: A Comparison Based on Individual Photographs Provided by Internet UsersJournal of Nonverbal Behavior, 2010
- Emotional expressivity as a signal of cooperationEvolution and Human Behavior, 2010
- Group-Based Trust in StrangersPsychological Science, 2009
- Relationship between Smiling and Laughter in Humans (Homo sapiens): Testing the Power Asymmetry HypothesisFolia Primatologica, 2008
- Social Functions of Emotions at Four Levels of AnalysisCognition and Emotion, 1999
- Trust, Reciprocity, and Social HistoryGames and Economic Behavior, 1995
- Two-day-olds prefer their native languageInfant Behavior and Development, 1993
- Expressiveness and Expressive ControlPersonality and Social Psychology Bulletin, 1992
- Hearing lips and seeing voicesNature, 1976