Layer-Parallel Training of Deep Residual Neural Networks
Authors:
- Stefanie GüntherSGStefanie Günther
- Lars RuthottoLRLars Ruthotto
- Jacob B. SchroderJSJacob B. Schroder
- Eric C. CyrECEric C. Cyr
- Nicolas R. GaugerNGNicolas R. Gauger
- 1 January 2020
- journal article
- research article
- Published by Society for Industrial & Applied Mathematics (SIAM) in SIAM Journal on Mathematics of Data Science
- Vol. 2 (1) , 1-23
Abstract
Residual neural networks (ResNets) are a promising class of deep neural networks that have shown excellent performance for a number of learning tasks, e.g., image classification and recognition. Ma...Keywords
Funding Information
- National Nuclear Security Administration (DE-NA0003525)
- National Science Foundation (DMS 1522599, DMS 1751636)
This publication has 25 references indexed in Scilit:
- Parallel Time Integration with MultigridSIAM Journal on Scientific Computing, 2014
- Adaptive sequencing of primal, dual, and design steps in simulation based optimizationComputational Optimization and Applications, 2013
- Adaptive Multilevel Inexact SQP Methods for PDE-Constrained OptimizationSIAM Journal on Optimization, 2011
- Approximate Nullspace Iterations for KKT SystemsSIAM Journal on Matrix Analysis and Applications, 2010
- Learning Deep Architectures for AIFoundations and Trends in Machine Learning, 2009
- Minimal Repetition Dynamic Checkpointing Algorithm for Unsteady Adjoint CalculationSIAM Journal on Scientific Computing, 2009
- Analysis of the Parareal Time‐Parallel Time‐Integration MethodSIAM Journal on Scientific Computing, 2007
- Parallel Lagrange--Newton--Krylov--Schur Methods for PDE-Constrained Optimization. Part I: The Krylov--Schur SolverSIAM Journal on Scientific Computing, 2005
- Gradient-based learning applied to document recognitionProceedings of the IEEE, 1998
- Multi-level adaptive solutions to boundary-value problemsMathematics of Computation, 1977