A Systematic Cross-Corpus Analysis of Human Reactions to Robot Conversational Failures
Authors:
Open Access
- 18 October 2021
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 112-120
Abstract
In this paper, we analyze multimodal behavioral responses to robot failures across different tasks. Two multimodal datasets are examined in which humans interact with guided-task robots in task-oriented dialogues. In both datasets, the robots simulated failures of conversational breakdown and miscommunication typically observed in human-robot interactions. We closely examine human reactions to these failures looking at facial and acoustic features. Our analyses identify the significant behavioral features for automatic detection of such failures in interaction. We also examine human responses to different types of robot failures and if failures occurred early or late in the interaction cause variation in the responses. Our findings indicate that several nonverbal behaviors are consistently present in responses to robots’ failures, e.g., gaze and speech prosody, whereas, linguistic features appear to be task-dependent. We discuss how these findings may generalize to other tasks, and how autonomous robots may identify opportunities to detect and recover from failures in interactions with humans.Keywords
Funding Information
- U.S. Army Research Office (W911NF-20-2-0053)
- Swedish Foundation for Strategic Research (GMT14-0082)
This publication has 22 references indexed in Scilit:
- OpenFace: An open source facial behavior analysis toolkit2016
- Fitting Linear Mixed-Effects Models Usinglme42015
- Automatic Detection of Miscommunication in Spoken Dialogue Systems2015
- A Model of Distributional Handing Interaction for a Mobile Robot2013
- Exploring human error recovery strategies: Implications for spoken dialogue systemsSpeech Communication, 2005
- Speech technology on trial: Experiences from the August systemNatural Language Engineering, 2000
- Automatic detection of poor speech recognition at the dialogue level1999
- Computers are social actors1994
- Nonverbal expectancy violations: Model elaboration and application to immediacy behaviorsCommunication Monographs, 1988
- Opening up ClosingsSemiotica, 1973