Discounted Continuous-Time Markov Decision Processes with Unbounded Rates: The Convex Analytic Approach
Authors:
- Alexey PiunovskiyAPAlexey Piunovskiy
- Yi ZhangYZYi Zhang
- 1 January 2011
- journal article
- research article
- Published by Society for Industrial & Applied Mathematics (SIAM) in SIAM Journal on Control and Optimization
- Vol. 49 (5) , 2032-2061
Abstract
This paper deals with constrained discounted continuous-time Markov decision processes, also known as controlled jump Markov processes, with Borel state and action spaces. Under some conditions imposed on the primitives, allowing unbounded transition rates and unbounded (from both above and below) cost rates, first, we study the space of occupation measures. Then we reformulate the original problem as a linear program over the space of those measures and undertake the duality analysis. Finally, under some compactness-continuity conditions, we show the existence of a stationary optimal policy out of the class of randomized history-dependent policies.Keywords
All Related Versions
This publication has 20 references indexed in Scilit:
- Discounted Continuous-Time Markov Decision Processes with Constraints: Unbounded Transition and Loss RatesMathematics of Operations Research, 2011
- New discount and average optimality conditions for continuous-time Markov decision processesAdvances in Applied Probability, 2010
- Towards the optimal control of Markov chains with constraintsAutomatica, 2010
- Continuous-Time Markov Decision Processes with Discounted Rewards: The Case of Polish SpacesMathematics of Operations Research, 2007
- A survey of recent results on continuous-time Markov decision processesTOP, 2006
- Constrained Continuous-Time Markov Control Processes with Discounted CriteriaStochastic Analysis and Applications, 2003
- A Controlled Jump Discounted Model with ConstraintsTheory of Probability & Its Applications, 1998
- Semi-Markov and Jump Markov Controlled Models: Average Cost CriterionTheory of Probability & Its Applications, 1986
- Multivariate point processes: predictable projection, Radon-Nikodym derivatives, representation of martingalesProbability Theory and Related Fields, 1975
- Continuously Discounted Markov Decision Model with Countable State and Action SpaceThe Annals of Mathematical Statistics, 1971