A trained humanoid robot can perform human-like crossmodal social attention and conflict resolution
Preprint
Abstract
To enhance human-robot social interaction, it is essential for robots to process multiple social cues in a complex real-world environment. However, incongruency of input information across modalities is inevitable and could be challenging for robots to process. To tackle this challenge, our study adopted the neurorobotic paradigm of crossmodal conflict resolution to make a robot express human-like social attention. A behavioural experiment was conducted on 37 participants for the human study. We designed a round-table meeting scenario with three animated avatars to improve ecological validity. Each avatar wore a medical mask to obscure the facial cues of the nose, mouth, and jaw. The central avatar shifted its eye gaze while the peripheral avatars generated sound. Gaze direction and sound locations were either spatially congruent or incongruent. We observed that the central avatar's dynamic gaze could trigger crossmodal social attention responses. In particular, human performances are better under the congruent audio-visual condition than the incongruent condition. Our saliency prediction model was trained to detect social cues, predict audio-visual saliency, and attend selectively for the robot study. After mounting the trained model on the iCub, the robot was exposed to laboratory conditions similar to the human experiment. While the human performances were overall superior, our trained model demonstrated that it could replicate attention responses similar to humans.Keywords
All Related Versions
This publication has 65 references indexed in Scilit:
- Do I Have a Personality? Endowing Care Robots with Context-Dependent Personality TraitsInternational Journal of Social Robotics, 2020
- Gated multimodal networksNeural Computing and Applications, 2020
- Abnormal alpha modulation in response to human eye gaze predicts inattention severity in children with ADHDDevelopmental Cognitive Neuroscience, 2019
- It’s in the Eyes: The Engaging Role of Eye Contact in HRIInternational Journal of Social Robotics, 2019
- Cross-modal cueing effects of visuospatial attention on conscious somatosensory perceptionHeliyon, 2018
- Directing Eye Gaze Enhances Auditory Spatial Cue DiscriminationCurrent Biology, 2014
- When one sees what the other hears: Crossmodal attentional modulation for gazed and non-gazed upon auditory targetsConsciousness and Cognition, 2010
- Atypical eye contact in autism: Models, mechanisms and developmentNeuroscience & Biobehavioral Reviews, 2009
- Joint attention: Inferring what others perceive (and don’t perceive)Consciousness and Cognition, 2008
- Perception of static eye gaze direction facilitates subsequent early visual processingClinical Neurophysiology, 2004