Hmm, You Seem Confused! Tracking Interlocutor Confusion for Situated Task-Oriented HRI
Open Access
- 13 March 2023
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 142-151
Abstract
Our research seeks to develop a long-lasting and high-quality engagement between the user and the social robot, which in turn requires a more sophisticated alignment of the user and the system than is currently commonly available. Close monitoring of interlocutors' states, and we argue their confusion state in particular, and adjusting dialogue policies based on this state of confusion is needed for successful joint activity. In this paper, we present an initial study of human-robot conversation scenarios using a Pepper robot to investigate the confusion states of users. A Wizard-of-Oz (WoZ) HRI experiment is illustrated in detail with stimuli strategies to trigger confused states from interlocutors. For the collected data, we estimated emotions, head pose, and eye gaze, and these features were analysed against the silence duration time of the speech data and the post-study self-reported confusion states that are reported by participants. Our analysis found a significant relationship between confusion states and most of these features. We see these results as being particularly significant for multimodal situated dialogues for human-robot interaction and beyond.Keywords
Funding Information
- The Science Foundation Ireland Centre for Research Training in Machine Learning (18/CRT/6183)
This publication has 39 references indexed in Scilit:
- Comparing Models of Disengagement in Individual and Group Interactions2015
- Confusion and its dynamics during device comprehension with breakdown scenariosActa Psychologica, 2014
- Confusion can be beneficial for learningLearning and Instruction, 2014
- Human amygdala response to dynamic facial expressions of positive and negative surprise.Emotion, 2014
- Wizard of Oz Studies in HRI: A Systematic Review and New Reporting GuidelinesJournal of Human-Robot Interaction, 2012
- Confusion and complex learning during interactions with computer learning environmentsThe Internet and Higher Education, 2012
- Confusion and interest: The role of knowledge emotions in aesthetic experience.Psychology of Aesthetics, Creativity, and the Arts, 2010
- The weirdest people in the world?Behavioral and Brain Sciences, 2010
- IEMOCAP: interactive emotional dyadic motion capture databaseLanguage Resources and Evaluation, 2008
- A spatial model of engagement for a social robot2006