Stochastic Normalizing Flows
Preprint
Abstract
We introduce stochastic normalizing flows, an extension of continuous normalizing flows for maximum likelihood estimation and variational inference (VI) using stochastic differential equations (SDEs). Using the theory of rough paths, the underlying Brownian motion is treated as a latent variable and approximated, enabling efficient training of neural SDEs as random neural ordinary differential equations. These SDEs can be used for constructing efficient Markov chains to sample from the underlying distribution of a given dataset. Furthermore, by considering families of targeted SDEs with prescribed stationary distribution, we can apply VI to the optimization of hyperparameters in stochastic MCMC.Keywords
All Related Versions
This publication has 8 references indexed in Scilit:
- Unbiased Estimation with Square Root Convergence for SDE ModelsOperations Research, 2015
- Riemann Manifold Langevin and Hamiltonian Monte Carlo MethodsJournal of the Royal Statistical Society Series B: Statistical Methodology, 2011
- Differential Equations Driven by Rough Paths: An Approach via Discrete ApproximationApplied Mathematics Research eXpress, 2008
- Seeing the Wood for the Trees: A Critical Evaluation of Methods to Estimate the Parameters of Stochastic Differential EquationsJournal of Financial Econometrics, 2007
- Exact and Computationally Efficient Likelihood-Based Estimation for Discretely Observed Diffusion Processes (with Discussion)Journal of the Royal Statistical Society Series B: Statistical Methodology, 2006
- Differential equations driven by rough signalsRevista Matemática Iberoamericana, 1998
- Wong-Zakai approximations for stochastic differential equationsActa Applicandae Mathematicae, 1996
- On the Convergence of Ordinary Integrals to Stochastic IntegralsThe Annals of Mathematical Statistics, 1965