Average Continuous Control of Piecewise Deterministic Markov Processes
Authors:
- O. L. V. CostaOCO. L. V. Costa
- F. DufourFDF. Dufour
- 1 January 2010
- journal article
- research article
- Published by Society for Industrial & Applied Mathematics (SIAM) in SIAM Journal on Control and Optimization
- Vol. 48 (7) , 4262-4291
Abstract
This paper deals with the long run average continuous control problem of piecewise deterministic Markov processes (PDMPs) taking values in a general Borel space and with compact action space depending on the state variable. The control variable acts on the jump rate and transition measure of the PDMP, and the running and boundary costs are assumed to be positive but not necessarily bounded. Our first main result is to obtain an optimality equation for the long run average cost in terms of a discrete-time optimality equation related to the embedded Markov chain given by the postjump location of the PDMP. Our second main result guarantees the existence of a feedback measurable selector for the discrete-time optimality equation by establishing a connection between this equation and an integro-differential equation. Our final main result is to obtain some sufficient conditions for the existence of a solution for a discrete-time optimality inequality and an ordinary optimal feedback control for the long run average cost using the so-called vanishing discount approach. Two examples are presented illustrating the possible applications of the results developed in the paper.Keywords
This publication has 34 references indexed in Scilit:
- Stability and Ergodicity of Piecewise Deterministic Markov ProcessesSIAM Journal on Control and Optimization, 2008
- Continuous Time Discounted Jump Markov Decision Processes: A Discrete-Event ApproachMathematics of Operations Research, 2004
- Piecewise Deterministic Markov Control Processes with Feedback Controls and Unbounded CostsActa Applicandae Mathematicae, 2004
- A Dynamic Programming Algorithm for the Optimal Control of Piecewise Deterministic Markov ProcessesSIAM Journal on Control and Optimization, 2001
- Stability of Piecewise-Deterministic Markov ProcessesSIAM Journal on Control and Optimization, 1999
- Generalized Bellman--Hamilton--Jacobi Optimality Conditions for a Control Problem with a Boundary ConditionApplied Mathematics & Optimization, 1996
- Discrete-Time Controlled Markov Processes with Average Cost Criterion: A SurveySIAM Journal on Control and Optimization, 1993
- Necessary and sufficient optimality conditions for control of piecewise deterministic markov processesStochastics and Stochastic Reports, 1992
- Average Impulse Control of Piecewise Deterministic ProcessesIMA Journal of Mathematical Control and Information, 1989
- Optimal capacity expansion under uncertaintyAdvances in Applied Probability, 1987