Deep-Energy: Unsupervised Training of Deep Neural Networks
Preprint
Abstract
The success of deep learning has been due, in no small part, to the availability of large annotated datasets. Thus, a major bottleneck in current learning pipelines is the time-consuming human annotation of data. In scenarios where such input-output pairs cannot be collected, simulation is often used instead, leading to a domain-shift between synthesized and real-world data. This work offers an unsupervised alternative that relies on the availability of task-specific energy functions, replacing the generic supervised loss. Such energy functions are assumed to lead to the desired label as their minimizer given the input. The proposed approach, termed "Deep Energy", trains a Deep Neural Network (DNN) to approximate this minimization for any chosen input. Once trained, a simple and fast feed-forward computation provides the inferred label. This approach allows us to perform unsupervised training of DNNs with real-world inputs only, and without the need for manually-annotated labels, nor synthetically created data. "Deep Energy" is demonstrated in this paper on three different tasks -- seeded segmentation, image matting and single image dehazing -- exposing its generality and wide applicability. Our experiments show that the solution provided by the network is often much better in quality than the one obtained by a direct minimization of the energy function, suggesting an added regularization property in our scheme.Keywords
All Related Versions
This publication has 19 references indexed in Scilit:
- Gated Fusion Network for Single Image Dehazing2018
- 3D deeply supervised network for automated segmentation of volumetric medical imagesMedical Image Analysis, 2017
- AOD-Net: All-in-One Dehazing Network2017
- Single Image Dehazing via Multi-scale Convolutional Neural Networks2016
- Non-local Image Dehazing2016
- Efficient Image Dehazing with Boundary Constraint and Contextual Regularization2013
- The Pascal Visual Object Classes (VOC) ChallengeInternational Journal of Computer Vision, 2009
- Visibility in bad weather from a single image2008
- Restoration of a single superresolution image from several blurred, noisy, and undersampled measured imagesIEEE Transactions on Image Processing, 1997
- A multiscale retinex for bridging the gap between color images and the human observation of scenesIEEE Transactions on Image Processing, 1997