Regularized Fitted Q-Iteration for planning in continuous-space Markovian decision problems
Authors:
- 1 January 2009
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- No. 07431619,p. 725-730
Abstract
Reinforcement learning with linear and non-linear function approximation has been studied extensively in the last decade. However, as opposed to other fields of machine learning such as supervised learning, the effect of finite sample has not been thoroughly addressed within the reinforcement learning framework. In this paper we propose to use L2 regularization to control the complexity of the value function in reinforcement learning and planning problems. We consider the Regularized Fitted Q-Iteration algorithm and provide generalization bounds that account for small sample sizes. Finally, a realistic visual-servoing problem is used to illustrate the benefits of using the regularization procedure.Keywords
This publication has 13 references indexed in Scilit:
- Empirical Bernstein stopping2008
- Learning near-optimal policies with Bellman-residual minimization based fitted policy iteration and a single sample pathMachine Learning, 2007
- Analyzing feature generation for value-function approximation2007
- Sparse Temporal Difference Learning Using LASSO2007
- EGT for multiple view geometry and visual servoing - Robotics and vision with pinhole and panoramics camerasIEEE Robotics & Automation Magazine, 2005
- Basis Function Adaptation in Temporal Difference Reinforcement LearningAnnals of Operations Research, 2005
- Reinforcement learning with Gaussian processes2005
- Capacity of reproducing kernel spaces in learning theoryIEEE Transactions on Information Theory, 2003
- A Distribution-Free Theory of Nonparametric Regression2002
- A robotics toolbox for MATLABIEEE Robotics & Automation Magazine, 1996