Situated Human–Robot Collaboration: predicting intent from grounded natural language
- 1 October 2018
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 827-833
Abstract
Research in human teamwork shows that a key element of fluid and fluent interactions is the interpretation of implicit verbal and non-verbal cues in context. This poses an issue to robotic platforms, however, as they have historically worked best when controlled through explicit commands that have employed structured, unequivocal representations of the external world and their human partners. In this work, we present a framework for effectively grounding situated and naturalistic speech to action selection during human-robot collaborative activities. This is accomplished by maintaining and incrementally updating separate “speech” and “context” models that jointly classify a collaborator's utterance. We evaluate the efficacy of the system on a collaborative construction task with an autonomous robot and human participants. We first demonstrate that our system is capable of acquiring and deploying new task representations from limited and naturalistic data sets, and without any prior domain knowledge of language or the task itself. Finally, we show that our system is capable of significantly improving performance on an unfamiliar task after a one-shot exposure.Keywords
This publication has 24 references indexed in Scilit:
- Transparent role assignment and task allocation in human robot collaboration2017
- Multisensory teamwork: using a tactile or an auditory display to exchange gaze information improves performance in joint visual searchErgonomics, 2015
- Automatic generation and detection of highly reliable fiducial markers under occlusionPattern Recognition, 2014
- Learning spatial-semantic representations from natural language descriptions and scene classifications2014
- A natural language planner interface for mobile manipulators2014
- Grounding spatial relations for human-robot interaction2013
- Learning perceptually grounded word meanings from unaligned parallel dataMachine Learning, 2013
- IkeaBot: An autonomous multi-robot coordinated furniture assembly system2013
- Towards quantitative modeling of task confirmations in human-robot dialog2011
- Lexical entrainment in human-robot interaction: Can robots entrain human vocabulary?2009