Risk-sensitive Markov control processes
Preprint
Abstract
We introduce a general framework for measuring risk in the context of Markov control processes with risk maps on general Borel spaces that generalize known concepts of risk measures in mathematical finance, operations research and behavioral economics. Within the framework, applying weighted norm spaces to incorporate also unbounded costs, we study two types of infinite-horizon risk-sensitive criteria, discounted total risk and average risk, and solve the associated optimization problems by dynamic programming. For the discounted case, we propose a new discount scheme, which is different from the conventional form but consistent with the existing literature, while for the average risk criterion, we state Lyapunov-like stability conditions that generalize known conditions for Markov chains to ensure the existence of solutions to the optimality equation.Keywords
All Related Versions
This publication has 27 references indexed in Scilit:
- Optimality equations and inequalities in a class of risk-sensitive average cost Markov decision chainsMathematical Methods of Operations Research, 2009
- The Perron-Frobenius theorem for homogeneous, monotone functionsTransactions of the American Mathematical Society, 2004
- Convex measures of risk and trading constraintsFinance and Stochastics, 2002
- Mixed risk-neutral/minimax control of discrete-time, finite-state Markov decision processesIEEE Transactions on Automatic Control, 2000
- Coherent Measures of RiskMathematical Finance, 1999
- From stochastic dominance to mean-risk models: Semideviations as risk measuresEuropean Journal of Operational Research, 1999
- Further Topics on Discrete-Time Markov Control Processes1999
- Risk sensitive control of Markov processes in countable state spaceSystems & Control Letters, 1996
- Advances in prospect theory: Cumulative representation of uncertaintyJournal of Risk and Uncertainty, 1992
- Recurrence conditions for Markov decision processes with Borel state space: A surveyAnnals of Operations Research, 1991