MAE: Mutual Posterior-Divergence Regularization for Variational AutoEncoders
Preprint
Abstract
Variational Autoencoder (VAE), a simple and effective deep generative model, has led to a number of impressive empirical successes and spawned many advanced variants and theoretical investigations. However, recent studies demonstrate that, when equipped with expressive generative distributions (aka. decoders), VAE suffers from learning uninformative latent representations with the observation called KL Varnishing, in which case VAE collapses into an unconditional generative model. In this work, we introduce mutual posterior-divergence regularization, a novel regularization that is able to control the geometry of the latent space to accomplish meaningful representation learning, while achieving comparable or superior capability of density estimation. Experiments on three image benchmark datasets demonstrate that, when equipped with powerful decoders, our model performs well both on density estimation and representation learning.Keywords
All Related Versions
This publication has 4 references indexed in Scilit:
- A Hierarchical Latent Variable Encoder-Decoder Model for Generating DialoguesProceedings of the AAAI Conference on Artificial Intelligence, 2017
- Representation Learning: A Review and New PerspectivesIEEE Transactions on Pattern Analysis and Machine Intelligence, 2013
- Gradient-based learning applied to document recognitionProceedings of the IEEE, 1998
- The "Wake-Sleep" Algorithm for Unsupervised Neural NetworksScience, 1995