Forward-reverse EM algorithm for Markov chains: convergence and numerical analysis
Preprint
Abstract
We develop a forward-reverse EM (FREM) algorithm for estimating parameters that determine the dynamics of a discrete time Markov chain evolving through a certain measurable state space. As a key tool for the construction of the FREM method we develop forward-reverse representations for Markov chains conditioned on a certain terminal state. These representations may be considered as an extension of the earlier work Bayer and Schoenmakers [2013] on conditional diffusions. We proof almost sure convergence of our algorithm for a Markov chain model with curved exponential family structure. On the numerical side we give a complexity analysis of the forward-reverse algorithm by deriving its expected cost. Two application examples are discuss to demonstrate the scope of possible applications ranging from models based on continuous time processes to discrete time Markov chain models.Keywords
All Related Versions
This publication has 16 references indexed in Scilit:
- Simulation of forward-reverse stochastic representations for conditional diffusionsThe Annals of Applied Probability, 2014
- Maximum likelihood estimation of mark–recapture–recovery models in the presence of continuous covariatesThe Annals of Applied Statistics, 2013
- On Convergence Properties of the Monte Carlo EM Algorithm2013
- Forward and reverse representations for Markov chainsStochastic Processes and their Applications, 2007
- Simulation of conditioned diffusion and application to parameter estimationStochastic Processes and their Applications, 2006
- Evaluation of conditional Wiener integrals by numerical integration of stochastic differential equationsJournal of Computational Physics, 2004
- A Gradient Algorithm Locally Equivalent to the Em AlgorithmJournal of the Royal Statistical Society Series B: Statistical Methodology, 1995
- Convergence and robustness of the Robbins-Monro algorithm truncated at randomly varying boundsStochastic Processes and their Applications, 1987
- Expected Length of the Longest Probe Sequence in Hash Code SearchingJournal of the ACM, 1981
- Maximum Likelihood from Incomplete Data Via the EM AlgorithmJournal of the Royal Statistical Society Series B: Statistical Methodology, 1977