Complexity control by gradient descent in deep networks

Authors:

Abstract
Overparametrized deep networks predict well, despite the lack of an explicit complexity control during training, such as an explicit regularization term. For exponential-type loss functions, we solve this puzzle by showing an effective regularization effect of gradient descent in terms of the normalized weights that are relevant for classification.
Funding Information
  • NSF | NSF Office of the Director | Office of International Science and Engineering (CCF-1231216, CCF-1231216, CCF-1231216)
  • JUMP Semiconductor Research Corporation (SRC) program sponsored by DARPA
  • NSF | NSF Office of the Director | Office of International Science and Engineering
  • NSF | NSF Office of the Director | Office of International Science and Engineering

This publication has 3 references indexed in Scilit: