Proximal Alternating Direction Network: A Globally Converged Deep Unrolling Framework
Preprint
Abstract
Deep learning models have gained great success in many real-world applications. However, most existing networks are typically designed in heuristic manners, thus lack of rigorous mathematical principles and derivations. Several recent studies build deep structures by unrolling a particular optimization model that involves task information. Unfortunately, due to the dynamic nature of network parameters, their resultant deep propagation networks do \emph{not} possess the nice convergence property as the original optimization scheme does. This paper provides a novel proximal unrolling framework to establish deep models by integrating experimentally verified network architectures and rich cues of the tasks. More importantly, we \emph{prove in theory} that 1) the propagation generated by our unrolled deep model globally converges to a critical-point of a given variational energy, and 2) the proposed framework is still able to learn priors from training data to generate a convergent propagation even when task information is only partially available. Indeed, these theoretical results are the best we can ask for, unless stronger assumptions are enforced. Extensive experiments on various real-world applications verify the theoretical convergence and demonstrate the effectiveness of designed deep models.Keywords
All Related Versions
This publication has 6 references indexed in Scilit:
- ImageNet classification with deep convolutional neural networksCommunications of the ACM, 2017
- Robust Image and Video Dehazing with Visual Artifact Suppression via Gradient Residual Minimization2016
- Non-local Image Dehazing2016
- Relaxed Majorization-Minimization for Non-Smooth and Non-Convex OptimizationProceedings of the AAAI Conference on Artificial Intelligence, 2016
- A block coordinate variable metric forward–backward algorithmJournal of Global Optimization, 2016
- Single Image Haze Removal Using Dark Channel PriorIEEE Transactions on Pattern Analysis and Machine Intelligence, 2010