Markov decision processes with risk-sensitive criteria: an overview
Top Cited Papers
Open Access
- 1 April 2024
- journal article
- research article
- Published by Springer Nature in Mathematical Methods of Operations Research
- Vol. 99 (1-2) , 141-178
Abstract
The paper provides an overview of the theory and applications of risk-sensitive Markov decision processes. The term ’risk-sensitive’ refers here to the use of the Optimized Certainty Equivalent as a means to measure expectation and risk. This comprises the well-known entropic risk measure and Conditional Value-at-Risk. We restrict our considerations to stationary problems with an infinite time horizon. Conditions are given under which optimal policies exist and solution procedures are explained. We present both the theory when the Optimized Certainty Equivalent is applied recursively as well as the case where it is applied to the cumulated reward. Discounted as well as non-discounted models are reviewed.Keywords
All Related Versions
Funding Information
- Karlsruher Institut für Technologie (KIT)
This publication has 151 references indexed in Scilit:
- Reinforcement Learning and Stochastic Optimization2022
- Markov Decision Processes with Applications to Finance2011
- Optimality equations and inequalities in a class of risk-sensitive average cost Markov decision chainsMathematical Methods of Operations Research, 2009
- Dynamic mean-risk optimization in a binomial modelMathematical Methods of Operations Research, 2008
- Risk-sensitive capacity control in revenue managementMathematical Methods of Operations Research, 2006
- Conditional Decision Processes with Recursive FunctionJournal of Mathematical Analysis and Applications, 1999
- Large Deviations Techniques and Applications1998
- Connections between stochastic control and dynamic gamesMathematics of Control, Signals, and Systems, 1996
- Discrete-Time Markov Control Processes1996
- Conditions for optimality in dynamic programming and for the limit of n-stage optimal policies to be optimalProbability Theory and Related Fields, 1975