Minimum Average Value-at-Risk for Finite Horizon Semi-Markov Decision Processes in Continuous Time
Authors:
- Yonghui HuangYHYonghui Huang
- Xianping GuoXGXianping Guo
- 1 January 2016
- journal article
- Published by Society for Industrial & Applied Mathematics (SIAM) in SIAM Journal on Optimization
- Vol. 26 (1) , 1-28
Abstract
This paper studies the average value-at-risk (AVaR) criterion for finite horizon semi-Markov decision processes (SMDPs) in continuous time. Via an alternative representation of AVaR, we reduce the problem of minimizing the AVaR of the finite horizon cost to two subproblems: one is to minimize the expected-positive-deviation of the finite horizon cost from some level over policies, which itself is a new and interesting problem in the finite horizon SMDP setting; the second is an ordinary problem of minimizing a function of a single variable. For the first subproblem, by the technique of extending the state space to include the cost level, we prove that the value function is a minimum solution to the optimality equation, and an optimal policy exists under suitable conditions. Furthermore, we show that the value function is the unique solution in a metric space to the optimality equation when one more condition is imposed, which plays a key role for the algorithm complexity analysis and the policy improvemen...Keywords
This publication has 19 references indexed in Scilit:
- A mean–variance optimization problem for discounted Markov decision processesEuropean Journal of Operational Research, 2012
- Markov Decision Processes with Average-Value-at-Risk criteriaMathematical Methods of Operations Research, 2011
- Finite horizon semi-Markov decision processes with application to maintenance systemsEuropean Journal of Operational Research, 2011
- Optimal risk probability for first passage models in semi-Markov decision processesJournal of Mathematical Analysis and Applications, 2009
- Variance minimization and the overtaking optimality approach to continuous-time controlled Markov chainsMathematical Methods of Operations Research, 2008
- Time Consistent Dynamic Risk MeasuresMathematical Methods of Operations Research, 2006
- Dynamic coherent risk measuresStochastic Processes and their Applications, 2004
- Credit risk optimization with Conditional Value-at-Risk criterionMathematical Programming, 2001
- Coherent Measures of RiskMathematical Finance, 1999
- Successive Approximations for Finite Horizon, Semi-Markov Decision Processes with Application to Asset LiquidationOperations Research, 1986