Counter examples for compact action markov decision chains with average reward criteria
- 1 January 1987
- journal article
- research article
- Published by Taylor & Francis in Communications in Statistics. Stochastic Models
- Vol. 3 (3) , 357-368
Abstract
In this note we present two examples of compact-action finite-state Markov decision chains in which a policy improvement procedure yields wrong or limited results. In the first example, which exhibits a multichain structure, there is no convergence of the average rewards of the successive policies to the maximal value. In the second example, which has a unichain structure, the lack of uniqueness of maximizing policies in each step of the algorithm means that there is no convergence of either bias vectors or maximizing policies. Accordingly, no solution to the average optimality equations can be obtained.Keywords
This publication has 3 references indexed in Scilit:
- On the Convergence of Policy Iteration in Finite State Undiscounted Markov Decision Processes: The Unichain CaseMathematics of Operations Research, 1987
- On undiscounted markovian decision processes with compact action spacesRAIRO - Operations Research, 1985
- On the Convergence of Policy Iteration in Stationary Dynamic ProgrammingMathematics of Operations Research, 1979