Likelihood ratio gradient estimation for stochastic recursions
Authors:
- Peter W. GlynnPGPeter W. Glynn
- Pierre L'ECuyerPLPierre L'ECuyer
- 1 December 1995
- journal article
- Published by Cambridge University Press (CUP) in Advances in Applied Probability
- Vol. 27 (4) , 1019-1053
Abstract
In this paper, we develop mathematical machinery for verifying that a broad class of general state space Markov chains reacts smoothly to certain types of perturbations in the underlying transition structure. Our main result provides conditions under which the stationary probability measure of an ergodic Harris-recurrent Markov chain is differentiable in a certain strong sense. The approach is based on likelihood ratio ‘change-of-measure' arguments, and leads directly to a ‘likelihood ratio gradient estimator' that can be computed numerically.Keywords
This publication has 16 references indexed in Scilit:
- Stochastic Optimization by Simulation: Convergence Proofs for the GI/G/1 Queue in Steady-StateManagement Science, 1994
- Note—Pathwise Convexity and its Relation to Convergence of Time-Average DerivativesManagement Science, 1992
- A GSMP formalism for discrete event systemsProceedings of the IEEE, 1989
- Discrete-time conversion for simulating semi-Markov processesOperations Research Letters, 1986
- Using the QR Factorization and Group Inversion to Compute, Differentiate, and Estimate the Sensitivity of Stationary Probabilities for Markov ChainsSIAM Journal on Algebraic Discrete Methods, 1986
- Continuity of Generalized Semi-Markov ProcessesMathematics of Operations Research, 1980
- A splitting technique for Harris recurrent Markov chainsProbability Theory and Related Fields, 1978
- A new approach to the limit theory of recurrent Markov chainsTransactions of the American Mathematical Society, 1978
- The continuity of the single server queueJournal of Applied Probability, 1972
- Perturbation theory and finite Markov chainsJournal of Applied Probability, 1968