On the Poisson Equation for Piecewise-Deterministic Markov Processes
Authors:
- Oswaldo L. V. CostaOCOswaldo L. V. Costa
- François DufourFDFrançois Dufour
- 1 January 2003
- journal article
- Published by Society for Industrial & Applied Mathematics (SIAM) in SIAM Journal on Control and Optimization
- Vol. 42 (3) , 985-1001
Abstract
In this paper we study the problem of the existence of a solution for the Poisson equation (PE) associated to a piecewise-deterministic Markov process (PDP). It is well known that the long run average cost of a stochastic process can be obtained through a solution of the PE associated with the process. Our first result will show that the existence of a solution for the PE of a PDP is equivalent to the existence of a solution for the PE of an embedded discrete- time Markov chain associated with the PDP. It is important to point out that, due to the possibility of jumps from the boundary, the differential formula for the PDPs has a special form, so that general results for the PE of continuous- time stochastic processes cannot be directly applied. Usually discrete-time conditions for the existence of a solution of a PE of a Markov chain are easier to apply than the continuous-time counterpart. We follow this approach to derive our second result, which establishes sufficient conditions for the existence of a PE to the embedded Markov chain, and consequently for the PE of the PDP. The condition is illustrated with an application to the capacity expansion model.Keywords
This publication has 7 references indexed in Scilit:
- Stability of Piecewise-Deterministic Markov ProcessesSIAM Journal on Control and Optimization, 1999
- A Liapounov bound for solutions of the Poisson equationThe Annals of Probability, 1996
- Discrete-Time Markov Control Processes1996
- State-Dependent Criteria for Convergence of Markov ChainsThe Annals of Applied Probability, 1994
- Markov Chains and Stochastic Stability1993
- Markov Models and Optimization1993
- Markov Processes1986