Correct Me If I'm Wrong: Using Non-Experts to Repair Reinforcement Learning Policies
Authors:
- 7 March 2022
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 493-501
Abstract
Reinforcement learning has shown great potential for learning sequential decision-making tasks. Yet, it is difficult to anticipate all possible real-world scenarios during training, causing robots to inevitably fail in the long run. Many of these failures are due to variations in the robot's environment. Usually experts are called to correct the robot's behavior; however, some of these failures do not necessarily require an expert to solve them. In this work, we query non-experts online for help and explore 1) if/how non-experts can provide feedback to the robot after a failure and 2) how the robot can use this feedback to avoid such failures in the future by generating shields that restrict or correct its high-level actions. We demonstrate our approach on common daily scenarios of a simulated kitchen robot. The results indicate that non-experts can indeed understand and repair robot failures. Our generated shields accelerate learning and improve data-efficiency during retraining.Keywords
Funding Information
- Swedish Research Council (2017–05189,2017–05102)
- Swedish Foundation for Strategic Research (SSF FFL18-0199)
This publication has 22 references indexed in Scilit:
- Supervised autonomy for online learning in human-robot interactionPattern Recognition Letters, 2017
- Simplified Programming of Re-usable Skills on a Safe Industrial Robot2017
- Robot programming by non-experts: Intuitiveness and robustness of One-Shot robot programming2016
- Semi-situated learning of verbal and nonverbal content for repeated human-robot interaction2016
- The hard lessons of DARPA's robotics challenge [News]IEEE Spectrum, 2015
- RoboFlow: A flow-based visual programming language for mobile manipulation tasks2015
- Robot Learning from Human TeachersSynthesis Lectures on Artificial Intelligence and Machine Learning, 2014
- Training a Robot via Human Feedback: A Case Study2013
- A survey of robot learning from demonstrationRobotics and Autonomous Systems, 2009
- Teachable robots: Understanding human teaching behavior to build more effective robot learnersArtificial Intelligence, 2008