A trained humanoid robot can perform human-like crossmodal social attention and conflict resolution
Authors:
- Di FuDFDi Fu
- Fares AbawiFAFares Abawi
- Hugo CarneiroHCHugo Carneiro
- Matthias KerzelMKMatthias Kerzel
- Ziwei ChenZCZiwei Chen
- Erik StrahlESErik Strahl
- Xun LiuXLXun Liu
- Stefan WermterSWStefan Wermter
Preprint
Abstract
Due to the aging population and life digitalisation, humanoid robots could be seen as potential community resources to accompany the elderly, support remote work, and improve individuals' mental and physical health. To enhance human-robot interaction, it is essential for robots to become more socialised via processing multiple social cues in a complex real-world environment. For this aim, our study adopted the neurorobotic paradigm of gaze-triggered audio-visual crossmodal conflict resolution to make an iCub robot express human-like social attention responses. For the human study, a behavioural experiment was conducted on 37 participants. To improve ecological validity, we designed a round-table meeting scenario with three animated avatars. Each avatar wore a medical mask to cover facial cues of the nose, mouth, and jaw. The central avatar was capable of gaze shifting, while the peripheral two were capable of sound generation. The gaze direction and the sound localisation were either congruent or incongruent. We observed that the central avatar's dynamic gaze could trigger crossmodal social attention with better human performance in the audio-visual congruent condition than in the incongruent condition. For the robot study, our saliency prediction model was trained to implement social cue detection, audio-visual saliency prediction, and selective attention. After finishing the model training, the iCub robot was exposed to similar laboratory conditions as the participants. While the human performance was overall superior, our trained model demonstrates that it can replicate similar attention responses as humans regarding the congruency and incongruency performance.Keywords
All Related Versions
This publication has 52 references indexed in Scilit:
- Preference for robot service or human service in hotels? Impacts of the COVID-19 pandemicInternational Journal of Hospitality Management, 2021
- Do I Have a Personality? Endowing Care Robots with Context-Dependent Personality TraitsInternational Journal of Social Robotics, 2020
- Gated multimodal networksNeural Computing and Applications, 2020
- Abnormal alpha modulation in response to human eye gaze predicts inattention severity in children with ADHDDevelopmental Cognitive Neuroscience, 2019
- It’s in the Eyes: The Engaging Role of Eye Contact in HRIInternational Journal of Social Robotics, 2019
- Cross-modal cueing effects of visuospatial attention on conscious somatosensory perceptionHeliyon, 2018
- Directing Eye Gaze Enhances Auditory Spatial Cue DiscriminationCurrent Biology, 2014
- When one sees what the other hears: Crossmodal attentional modulation for gazed and non-gazed upon auditory targetsConsciousness and Cognition, 2010
- Atypical eye contact in autism: Models, mechanisms and developmentNeuroscience & Biobehavioral Reviews, 2009
- Joint attention: Inferring what others perceive (and don’t perceive)Consciousness and Cognition, 2008