A Unifying Framework for Reinforcement Learning and Planning
Open Access
- 11 July 2022
- journal article
- research article
- Published by Frontiers Media SA in Frontiers in Artificial Intelligence
- Vol. 5, 908353
Abstract
Sequential decision making, commonly formalized as optimization of a Markov Decision Process, is a key challenge in artificial intelligence. Two successful approaches to MDP optimization arereinforcement learningandplanning, which both largely have their own research communities. However, if both research fields solve the same problem, then we might be able to disentangle the common factors in their solution approaches. Therefore, this paper presents a unifying algorithmic framework for reinforcement learning and planning (FRAP), which identifies underlying dimensions on which MDP planning and learning algorithms have to decide. At the end of the paper, we compare a variety of well-known planning, model-free and model-based RL algorithms along these dimensions. Altogether, the framework may help provide deeper insight in the algorithmic design space of planning and reinforcement learning.Keywords
All Related Versions
Funding Information
- Universiteit Leiden
This publication has 92 references indexed in Scilit:
- Active learning of inverse models with intrinsically motivated goal exploration in robotsRobotics and Autonomous Systems, 2013
- A Survey on Policy Search for RoboticsFoundations and Trends in Robotics, 2011
- $\hbox{MAWA}^{\ast}$—A Memory-Bounded Anytime Heuristic-Search AlgorithmIEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), 2010
- LAO∗: A heuristic search algorithm that finds solutions with loopsArtificial Intelligence, 2001
- Linear-space best-first searchArtificial Intelligence, 1993
- Heuristic search viewed as path finding in a graphArtificial Intelligence, 1970
- Dynamic ProgrammingScience, 1966
- A note on two problems in connexion with graphsNumerische Mathematik, 1959
- The theory of dynamic programmingBulletin of the American Mathematical Society, 1954
- ON THE LIKELIHOOD THAT ONE UNKNOWN PROBABILITY EXCEEDS ANOTHER IN VIEW OF THE EVIDENCE OF TWO SAMPLESBiometrika, 1933