Correct Me If I am Wrong: Interactive Learning for Robotic Manipulation
- 25 January 2022
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Robotics and Automation Letters
- Vol. 7 (2) , 3695-3702
Abstract
Learning to solve complex manipulation tasks from visual observations is a dominant challenge for real-world robot learning. Although deep reinforcement learning algorithms have recently demonstrated impressive results in this context, they still require an impractical amount of time-consuming trial-and-error iterations. In this work, we consider the promising alternative paradigm of interactive learning in which a human teacher provides feedback to the policy during execution, as opposed to imitation learning where a pre-collected dataset of perfect demonstrations is used. Our proposed CEILing (Corrective and Evaluative Interactive Learning) framework combines both corrective and evaluative feedback from the teacher to train a stochastic policy in an asynchronous manner, and employs a dedicated mechanism to trade off human corrections with the robot’s own experience. We present results obtained with our framework in extensive simulation and real-world experiments to demonstrate that CEILing can effectively solve complex robot manipulation tasks directly from raw images in less than one hour of real-world training.Keywords
All Related Versions
Funding Information
- BrainLinks-BrainTools Center
- Albert-Ludwigs-Universität Freiburg
This publication has 29 references indexed in Scilit:
- COACH: Learning continuous actions from COrrective Advice Communicated by Humans2015
- Learning preferences for manipulation tasks from online coactive feedbackThe International Journal of Robotics Research, 2015
- Online Trajectory Planning in Dynamic Environments for Surgical Task Automation2014
- Optimization and learning for rough terrain legged locomotionThe International Journal of Robotics Research, 2011
- Preference-Based Policy Learning2011
- Complementary humanoid behavior shaping using corrective demonstration2010
- Autonomous Helicopter Aerobatics through Apprenticeship LearningThe International Journal of Robotics Research, 2010
- TAMER: Training an Agent Manually via Evaluative Reinforcement2008
- Integrated learning for interactive synthetic characters2002
- Robotic clicker trainingRobotics and Autonomous Systems, 2002