SGD-Net: Efficient Model-Based Deep Learning With Theoretical Guarantees
- 1 June 2021
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Computational Imaging
- Vol. 7 (25730436) , 598-610
Abstract
Deep unfolding networks have recently gained popularity for solving imaging inverse problems. However, the computational and memory complexity of data-consistency layers within traditional deep unfolding networks scales with the number of measurements, limiting their applicability to large-scale imaging inverse problems. We propose SGD-Net as a new methodology for improving the efficiency of deep unfolding through stochastic approximations of the data-consistency layers. Our theoretical analysis shows that SGD-Net can be trained to approximate batch deep unfolding networks to an arbitrary precision. Our simulations on intensity diffraction tomography and sparse-view computed tomography show that SGD-Net can match the performance of the traditional batch network at a fraction of training and testing complexity.Keywords
All Related Versions
Funding Information
- NSF (CCF-1813910)
- Laboratory Directed Research and Development
- Los Alamos National Laboratory (20200061DR)
This publication has 74 references indexed in Scilit:
- Distributed Optimization and Statistical Learning via the Alternating Direction Method of MultipliersFoundations and Trends in Machine Learning, 2011
- Incremental proximal methods for large scale convex optimizationMathematical Programming, 2011
- Solving Constrained Total-variation Image Restoration and Reconstruction Problems via Alternating Direction MethodsSIAM Journal on Scientific Computing, 2010
- An iterative thresholding algorithm for linear inverse problems with a sparsity constraintCommunications on Pure and Applied Mathematics, 2004
- Image quality assessment: from error visibility to structural similarityIEEE Transactions on Image Processing, 2004
- An EM algorithm for wavelet-based image restorationIEEE Transactions on Image Processing, 2003
- Wavelet-based image estimation: an empirical Bayes approach using Jeffrey's noninformative priorIEEE Transactions on Image Processing, 2001
- Nonlinear total variation based noise removal algorithmsPhysica D: Nonlinear Phenomena, 1992
- On the Douglas—Rachford splitting method and the proximal point algorithm for maximal monotone operatorsMathematical Programming, 1992
- A Stochastic Approximation MethodThe Annals of Mathematical Statistics, 1951