Learning Human Preferences over a Human-Robot Collaboration Based on Explicit and Implicit Human Feedback
Open Access
- 16 March 2026
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 1040-1049
Abstract
There is significant interest in enabling robots to learn to perform tasks directly from interactions with non-expert users. Typically, a human serves as a teacher whose only task is to provide feedback to a robot learner. However, in real-world human-robot collaborations, the human often assists with the task while also offering feedback. Our key insight is that we can extract additional, implicit feedback from the human’s actions during the collaboration to augment the robot learning process. Under the assumption of fixed-role assignments, we first propose to formalize human preferences over a human-robot collaboration as a shared set of parameters encoding alignment between two reward functions: one that drives human behavior, and another that should direct robot behavior. This allows us to extract implicit feedback from an interaction by reasoning about the human’s actions in the task as actions that reveal the human’s preferences. Then, we combine this implicit feedback with traditional explicit human feedback to facilitate estimating the human’s preferences. We evaluated our proposed approach for Preference learning from Implicit and Explicit feedback (PIE) in simulations and with real users in a cooking scenario. Our simulation results indicate that combining multiple modalities of human feedback improves a robot’s ability to estimate human preferences over the collaboration, with a similar trend observed in real-world evaluations. These findings highlight a promising direction for enabling robots to adapt to a user’s preference model more quickly, thereby reducing the amount of time a person must spend teaching a robot.Keywords
Funding Information
- Tata Consultancy Services (Grant)
- Tata Sons (Grant)
- Titan (Grant)
- National Science Foundation (IIS-2143109)
This publication has 37 references indexed in Scilit:
- Dancing with service robots: The impacts of employee-robot collaboration on hotel employees’ job craftingInternational Journal of Hospitality Management, 2022
- MIND MELD: Personalized Meta-Learning for Robot-Centric Imitation Learning2022
- A Survey of Robot Learning Strategies for Human-Robot Collaboration in Industrial SettingsRobotics and Computer-Integrated Manufacturing, 2022
- Affective Robot Story-Telling Human-Robot Interaction: Exploratory Real-Time Emotion Estimation Analysis Using Facial Expressions and Physiological SignalsIEEE Access, 2020
- MoveIt! Task Constructor for Task-Level Motion Planning2019
- Socially Assistive RoboticsIEEE Robotics & Automation Magazine, 2011
- Robot reinforcement learning using EEG-based reward signals2010
- Action understanding as inverse planningCognition, 2009
- Footing in human-robot conversations2009
- Teachable robots: Understanding human teaching behavior to build more effective robot learnersArtificial Intelligence, 2008