Likelihood Training of Cascaded Diffusion Models via Hierarchical Volume-preserving Maps
Preprint
Abstract
Cascaded models are multi-scale generative models with a marked capacity for producing perceptually impressive samples at high resolutions. In this work, we show that they can also be excellent likelihood models, so long as we overcome a fundamental difficulty with probabilistic multi-scale models: the intractability of the likelihood function. Chiefly, in cascaded models each intermediary scale introduces extraneous variables that cannot be tractably marginalized out for likelihood evaluation. This issue vanishes by modeling the diffusion process on latent spaces induced by a class of transformations we call hierarchical volume-preserving maps, which decompose spatially structured data in a hierarchical fashion without introducing local distortions in the latent space. We demonstrate that two such maps are well-known in the literature for multiscale modeling: Laplacian pyramids and wavelet transforms. Not only do such reparameterizations allow the likelihood function to be directly expressed as a joint likelihood over the scales, we show that the Laplacian pyramid and wavelet transform also produces significant improvements to the state-of-the-art on a selection of benchmarks in likelihood modeling, including density estimation, lossless compression, and out-of-distribution detection. Investigating the theoretical basis of our empirical gains we uncover deep connections to score matching under the Earth Mover's Distance (EMD), which is a well-known surrogate for perceptual similarity. Code can be found at \href{https://github.com/lihenryhfl/pcdm}{this https url}.Keywords
All Related Versions
This publication has 9 references indexed in Scilit:
- Elucidating the Design Space of Diffusion-Based Generative Models2022
- Efficient Content-Based Sparse Attention with Routing TransformersTransactions of the Association for Computational Linguistics, 2021
- MoverScore: Text Generation Evaluating with Contextualized Embeddings and Earth Mover Distance2019
- A Connection Between Score Matching and Denoising AutoencodersNeural Computation, 2011
- Consistency of Pseudolikelihood Estimation of Fully Visible Boltzmann MachinesNeural Computation, 2006
- The Earth Mover's Distance as a Metric for Image RetrievalInternational Journal of Computer Vision, 2000
- The Change-of-Variables Formula Using Matrix VolumeSIAM Journal on Matrix Analysis and Applications, 1999
- A Wavelet Tour of Signal Processing1999
- The "Wake-Sleep" Algorithm for Unsupervised Neural NetworksScience, 1995