Addressing Variance Shrinkage in Variational Autoencoders using Quantile Regression
Preprint
Abstract
Estimation of uncertainty in deep learning models is of vital importance, especially in medical imaging, where reliance on inference without taking into account uncertainty could lead to misdiagnosis. Recently, the probabilistic Variational AutoEncoder (VAE) has become a popular model for anomaly detection in applications such as lesion detection in medical images. The VAE is a generative graphical model that is used to learn the data distribution from samples and then generate new samples from this distribution. By training on normal samples, the VAE can be used to detect inputs that deviate from this learned distribution. The VAE models the output as a conditionally independent Gaussian characterized by means and variances for each output dimension. VAEs can therefore use reconstruction probability instead of reconstruction error for anomaly detection. Unfortunately, joint optimization of both mean and variance in the VAE leads to the well-known problem of shrinkage or underestimation of variance. We describe an alternative approach that avoids this variance shrinkage problem by using quantile regression. Using estimated quantiles to compute mean and variance under the Gaussian assumption, we compute reconstruction probability as a principled approach to outlier or anomaly detection. Results on simulated and Fashion MNIST data demonstrate the effectiveness of our approach. We also show how our approach can be used for principled heterogeneous thresholding for lesion detection in brain images.Keywords
All Related Versions
This publication has 8 references indexed in Scilit:
- Validating Uncertainty in Medical Image Translation2020
- Beyond Expectation: Deep Joint Mean and Quantile Regression for Spatiotemporal ProblemsIEEE Transactions on Neural Networks and Learning Systems, 2020
- ISLES 2015 - A public evaluation benchmark for ischemic stroke lesion segmentation from multispectral MRIMedical Image Analysis, 2017
- Modeling and Transforming Speech Using Variational Autoencoders2016
- Transforming Research and Clinical Knowledge in Traumatic Brain Injury Pilot: Multicenter Implementation of the Common Data Elements for Traumatic Brain InjuryJournal of Neurotrauma, 2013
- Investigation of Prognostic Ability of Novel Imaging Markers for Traumatic Brain Injury (TBI)2011
- Divergence Estimation for Multidimensional Densities Via $k$-Nearest-Neighbor DistancesIEEE Transactions on Information Theory, 2009
- Controlling the False Discovery Rate: A Practical and Powerful Approach to Multiple TestingJournal of the Royal Statistical Society Series B: Statistical Methodology, 1995