Optimal Control of Probability on a Target Set for Continuous-Time Markov Chains
- 22 May 2023
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Automatic Control
- Vol. 69 (2) , 1202-1209
Abstract
In this paper, a stochastic optimal control problem is considered for a continuous-time Markov chain taking values in a denumerable state space over a fixed finite horizon. The optimality criterion is the probability that the process remains in a target set before and at a certain time. The optimal value is a superadditive capacity of target sets. Under some minor assumptions for the controlled Markov process, we establish the dynamic programming principle, based on which we prove that the value function is a classical solution of the Hamilton-Jacobi-Bellman equation on a discrete lattice space. We then prove that there exists an optimal deterministic Markov control under the compactness assumption of control domain. We further prove that the value function is the unique solution of the HJB equation. We also consider the case starting from the outside of target set and give the corresponding results. Finally, we apply our results to two examples.Keywords
Funding Information
- National Key R&D Program of China (2020YFA0712400)
- Engineering and Physical Sciences Research Council (EP/S005293/2)
This publication has 30 references indexed in Scilit:
- Optimal dividend problems with a risk probability criterionNaval Research Logistics (NRL), 2021
- Two-player zero-sum stochastic differential games with regime switchingAutomatica, 2020
- Nonzero-Sum Stochastic Games with Probability CriteriaDynamic Games and Applications, 2019
- Optimal switching under a hybrid diffusion model and applications to stock tradingAutomatica, 2018
- The risk probability criterion for discounted continuous-time Markov decision processesDiscrete Event Dynamic Systems, 2017
- Markov Decision Processes with Applications to Finance2011
- Towards the optimal control of Markov chains with constraintsAutomatica, 2010
- Optimal risk probability for first passage models in semi-Markov decision processesJournal of Mathematical Analysis and Applications, 2009
- Optimization of queuing system via stochastic controlAutomatica, 2009
- Multicriteria impulsive control of jump Markov processesMathematical Methods of Operations Research, 2004