A Tutorial on Deep Latent Variable Models of Natural Language
Preprint
Abstract
There has been much recent, exciting work on combining the complementary strengths of latent variable models and deep learning. Latent variable modeling makes it easy to explicitly specify model constraints through conditional independence properties, while deep learning makes it possible to parameterize these conditional likelihoods with powerful function approximators. While these "deep latent variable" models provide a rich, flexible framework for modeling many real-world phenomena, difficulties exist: deep parameterizations of conditional likelihoods usually make posterior inference intractable, and latent variable objectives often complicate backpropagation by introducing points of non-differentiability. This tutorial explores these issues in depth through the lens of variational inference.Keywords
All Related Versions
This publication has 14 references indexed in Scilit:
- StackGAN: Text to Photo-Realistic Image Synthesis with Stacked Generative Adversarial Networks2017
- Variational Inference: A Review for StatisticiansJournal of the American Statistical Association, 2017
- Perturb-and-MAP random fields: Using discrete optimization to learn and sample from energy models2011
- A Fast Learning Algorithm for Deep Belief NetsNeural Computation, 2006
- Monte Carlo Methods in Financial Engineering2003
- The "Wake-Sleep" Algorithm for Unsupervised Neural NetworksScience, 1995
- Adaptive Mixtures of Local ExpertsNeural Computation, 1991
- Tagging text with a probabilistic model1991
- Recent trends in hierarchic document clustering: A critical reviewInformation Processing & Management, 1988
- Maximum Likelihood from Incomplete Data Via the EM AlgorithmJournal of the Royal Statistical Society Series B: Statistical Methodology, 1977