Denumerable-state continuous-time Markov decision processes with unbounded transition and reward rates under the discounted criterion
Authors:
- 1 March 2002
- journal article
- research article
- Published by Cambridge University Press (CUP) in Journal of Applied Probability
- Vol. 39 (02) , 233-250
Abstract
In this paper, we consider denumerable-state continuous-time Markov decision processes with (possibly unbounded) transition and reward rates and general action space under the discounted criterion. We provide a set of conditions weaker than those previously known and then prove the existence of optimal stationary policies within the class of all possibly randomized Markov policies. Moreover, the results in this paper are illustrated by considering the birth-and-death processes with controlled immigration in which the conditions in this paper are satisfied, whereas the earlier conditions fail to hold.Keywords
This publication has 16 references indexed in Scilit:
- Value iteration in a class of average controlled Markov chains with unbounded costs: necessary and sufficient conditions for pointwise convergenceJournal of Applied Probability, 1996
- Optimal control of random walks, birth and death processes, and queuesAdvances in Applied Probability, 1981
- On Homogeneous Markov Models with Continuous Time and Finite or Countable State SpaceTheory of Probability & Its Applications, 1979
- Controlled Markov Models with Countable State Space and Continuous TimeTheory of Probability & Its Applications, 1978
- Note—A Note on Dynamic Programming with Unbounded RewardsManagement Science, 1978
- Optimal stationary policies for denumerable Markov chains in continuous timeAdvances in Applied Probability, 1976
- On Maximal Rewards and $|varepsilon$-Optimal Policies in Continuous Time Markov Decision ChainsThe Annals of Statistics, 1974
- Continuously Discounted Markov Decision Model with Countable State and Action SpaceThe Annals of Mathematical Statistics, 1971
- Finite state continuous time Markov decision processes with an infinite planning horizonJournal of Mathematical Analysis and Applications, 1968
- On the integro-differential equations of purely discontinuous Markoff processesTransactions of the American Mathematical Society, 1940