A review of inverse reinforcement learning theory and recent advances
Authors:
- 1 June 2012
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- No. 1089778X,p. 1-8
Abstract
A major challenge faced by machine learning community is the decision making problems under uncertainty. Reinforcement Learning (RL) techniques provide a powerful solution for it. An agent used by RL interacts with a dynamic environment and finds a policy through a reward function, without using target labels like Supervised Learning (SL). However, one fundamental assumption of existing RL algorithms is that reward function, the most succinct representation of the designer's intention, needs to be provided beforehand. In practice, the reward function can be very hard to specify and exhaustive to tune for large and complex problems, and this inspires the development of Inverse Reinforcement Learning (IRL), an extension of RL, which directly tackles this problem by learning the reward function through expert demonstrations. IRL introduces a new way of learning policies by deriving expert's intentions, in contrast to directly learning policies, which can be redundant and have poor generalization ability. In this paper, the original IRL algorithms and its close variants, as well as their recent advances are reviewed and compared.Keywords
This publication has 29 references indexed in Scilit:
- Perceptual Interpretation for Autonomous Navigation through Dynamic Imitation Learning2011
- Modified reward function on abstract features in inverse reinforcement learningJournal of Zhejiang University SCIENCE C, 2010
- Autonomous Helicopter Aerobatics through Apprenticeship LearningThe International Journal of Robotics Research, 2010
- Learning from Demonstration for Autonomous Navigation in Complex Unstructured TerrainThe International Journal of Robotics Research, 2010
- Learning to search: Functional gradient techniques for imitation learningAutonomous Robots, 2009
- Active Learning for Reward Estimation in Inverse Reinforcement Learning2009
- Autonomous Autorotation of an RC Helicopter2009
- Reinforcement Learning: An IntroductionIEEE Transactions on Neural Networks, 1998
- Efficient Training of Artificial Neural Networks for Autonomous NavigationNeural Computation, 1991
- Information Theory and Statistical Mechanics. IIPhysical Review B, 1957