Discrete-time singularly perturbed Markov chains: aggregation, occupation measures, and switching diffusion limit
Authors:
- G. YinGYG. Yin
- Q. ZhangQZQ. Zhang
- G. BadowskiGBG. Badowski
- 1 June 2003
- journal article
- Published by Cambridge University Press (CUP) in Advances in Applied Probability
- Vol. 35 (2) , 449-476
Abstract
This work is devoted to asymptotic properties of singularly perturbed Markov chains in discrete time. The motivation stems from applications in discrete-time control and optimization problems, manufacturing and production planning, stochastic networks, and communication systems, in which finite-state Markov chains are used to model large-scale and complex systems. To reduce the complexity of the underlying system, the states in each recurrent class are aggregated into a single state. Although the aggregated process may not be Markovian, its continuous-time interpolation converges to a continuous-time Markov chain whose generator is a function determined by the invariant measures of the recurrent states. Sequences of occupation measures are defined. A mean square estimate on a sequence of unscaled occupation measures is obtained. Furthermore, it is proved that a suitably scaled sequence of occupation measures converges to a switching diffusion.Keywords
This publication has 25 references indexed in Scilit:
- Asymptotically optimal controls of hybrid linear quadratic regulators in discrete timeAutomatica, 2002
- Asymptotic properties of a singularly perturbed Markov chain with inclusion of transient statesThe Annals of Applied Probability, 2000
- On nearly optimal controls of hybrid LQG problemsIEEE Transactions on Automatic Control, 1999
- Continuous-Time Markov Chains and Applications: A Singular Perturbation ApproachJournal of the American Statistical Association, 1998
- Statistical multiplexing of multiple time-scale Markov streamsIEEE Journal on Selected Areas in Communications, 1995
- Algorithms for singularly perturbed limiting average Markov control problemsIEEE Transactions on Automatic Control, 1992
- Approximation and Weak Convergence Methods for Random Processes, With Applications to Stochastic Systems Theory.Journal of the American Statistical Association, 1985
- A singular perturbation approach to modeling and control of Markov chainsIEEE Transactions on Automatic Control, 1981
- Singularly perturbed difference equations in optimal control problemsIEEE Transactions on Automatic Control, 1981
- Aggregation of Variables in Dynamic SystemsEconometrica, 1961