Reinforcement learning algorithms for semi-Markov decision processes with average reward
Author:
- 1 April 2012
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- Vol. 15, 157-162
Abstract
In this paper, we study reinforcement learning (RL) algorithms based on a perspective of performance sensitivity analysis for SMDPs with average reward. We present the results about performance sensitivity analysis for SMDPs with average reward. On these bases, two RL algorithms for average-reward SMDPs are studied. One algorithm is the relative value iteration (RVI) RL algorithm, which avoids the estimation of optimal average reward in the process of learning. Another algorithm is a policy gradient estimation algorithm, which extends the policy gradient estimation algorithm for discrete time Markov decision processes (MDPs) to SMDPs and only requires half storage of the existing algorithm.Keywords
This publication has 18 references indexed in Scilit:
- A policy gradient method for semi-Markov decision processes with application to call admission controlEuropean Journal of Operational Research, 2007
- Reinforcement learning for long-run average costEuropean Journal of Operational Research, 2004
- A Reinforcement Learning Algorithm Based on Policy Iteration for Average Reward: Empirical Results with Yield Management and Convergence AnalysisMachine Learning, 2004
- Semi-markov decision problems and performance sensitivity analysisIEEE Transactions on Automatic Control, 2003
- OnActor-Critic AlgorithmsSIAM Journal on Control and Optimization, 2003
- Simulation-based optimization of Markov reward processesIEEE Transactions on Automatic Control, 2001
- Learning Algorithms for Markov Decision Processes with Average CostSIAM Journal on Control and Optimization, 2001
- Average reward reinforcement learning: Foundations, algorithms, and empirical resultsMachine Learning, 1996
- Q-learningMachine Learning, 1992
- Simple statistical gradient-following algorithms for connectionist reinforcement learningMachine Learning, 1992