Successive approximations in partially observable controlled Markov chains with risk-sensitive average criterion
- 1 December 2005
- journal article
- research article
- Published by Taylor & Francis in Stochastics
- Vol. 77 (6) , 537-568
Abstract
Partially observable Markov decision chains with finite state, action and signal spaces are considered. The performance index is the risk-sensitive average criterion and, under conditions concerning reachability between the unobservable states and observability of the signals, it is shown that the value iteration algorithm can be implemented to approximate the optimal average cost, to determine a stationary policy whose performance index is arbitrarily close to the optimal one, and to establish the existence of solutions to the optimality equation. The results rely on an appropriate extension of the well-known Schweitzer's transformation.Keywords
This publication has 18 references indexed in Scilit:
- Small Parameter Limit for Ergodic, Discrete-Time, Partially Observed, Risk-Sensitive Control ProblemsMathematics of Control, Signals, and Systems, 2001
- The vanishing discount approach in Markov chains with risk-sensitive criteriaIEEE Transactions on Automatic Control, 2000
- Average Cost Dynamic Programming Equations For Controlled Markov Chains With Partial ObservationsSIAM Journal on Control and Optimization, 2000
- Risk sensitive control of finite state Markov chains in discrete time, with applications to portfolio managementMathematical Methods of Operations Research, 1999
- A pause control approach to the value iteration scheme in average Markov decision processesSystems & Control Letters, 1998
- A note on the convergence rate of the value iteration scheme in controlled Markov chainsSystems & Control Letters, 1998
- Small Parameter Limit for Discrete-Time Partially Observed Risk-Sensitive Control ProblemsSIAM Journal on Control and Optimization, 1998
- Discrete-Time Controlled Markov Processes with Average Cost Criterion: A SurveySIAM Journal on Control and Optimization, 1993
- Stochastic Control of Partially Observable Systems1992
- Optimal control of Markov processes with incomplete state informationJournal of Mathematical Analysis and Applications, 1965