Fundamentals of Artificial Neural Networks and Deep Learning

Authors:
  • MLMontesinos López
    Montesinos López
  • OAOsval Antonio
    Osval Antonio
  • ALAbelardo Montesinos López
    Abelardo Montesinos López
  • JCJose Crossa
    Jose Crossa

Top Cited Papers
Open Access
Abstract
In this chapter, we go through the fundamentals of artificial neural networks and deep learning methods. We describe the inspiration for artificial neural networks and how the methods of deep learning are built. We define the activation function and its role in capturing nonlinear patterns in the input data. We explain the universal approximation theorem for understanding the power and limitation of these methods and describe the main topologies of artificial neural networks that play an important role in the successful implementation of these methods. We also describe loss functions (and their penalized versions) and give details about in which circumstances each of them should be used or preferred. In addition to the Ridge, Lasso, and Elastic Net regularization methods, we provide details of the dropout and the early stopping methods. Finally, we provide the backpropagation method and illustrate it with two simple artificial neural networks.
Funding Information
  • Bill and Melinda Gates Foundation (INV-003439 BMGF/FCDO)

This publication has 25 references indexed in Scilit: