Necessary and Sufficient Conditions for Optimal Control of Semi-Markov Jump Processes
Author:
- Lawrence D. StoneLSLawrence D. Stone
- 1 May 1973
- journal article
- Published by Society for Industrial & Applied Mathematics (SIAM) in SIAM Journal on Control
- Vol. 11 (2) , 187-201
Abstract
A class of controlled semi-Markov jump processes is defined in this paper. Conditions are found which guarantee that satisfaction of the dynamic programming equations for stochastic control is necessary and sufficient for the minimization of the expected discounted cost of a controlled semi-Markov process over a random time. Terminal costs are included, and the controls are allowed to depend on both the present state of the process and the length of time the process has been in that state.Keywords
This publication has 13 references indexed in Scilit:
- Optimal Continuous-Parameter Stochastic ControlSIAM Review, 1969
- Continuous Time Markovian Sequential Control ProcessesSIAM Journal on Control, 1969
- Linear programming algorithms for semi-Markovian decision processesJournal of Mathematical Analysis and Applications, 1968
- Optimal Discounted Stochastic Control for Diffusion ProcessesSIAM Journal on Control, 1967
- Denumerable State Markovian Decision Processes-Average Cost CriterionThe Annals of Mathematical Statistics, 1966
- Markovian sequential control processes—Denumerable state spaceJournal of Mathematical Analysis and Applications, 1965
- Optimal control of Markov processes with incomplete state informationJournal of Mathematical Analysis and Applications, 1965
- Discounted Dynamic ProgrammingThe Annals of Mathematical Statistics, 1965
- Markov Processes1965
- Markov-Renewal Programming. I: Formulation, Finite Return ModelsOperations Research, 1963