Fundamentals of Artificial Neural Networks and Deep Learning
Authors:
- Montesinos LópezMLMontesinos López
- Osval AntonioOAOsval Antonio
- Abelardo Montesinos LópezALAbelardo Montesinos López
- Jose CrossaJCJose Crossa
Top Cited Papers
Open Access
- 14 January 2022
- book chapter
- Published by Springer Nature
- p. 379-425
Abstract
In this chapter, we go through the fundamentals of artificial neural networks and deep learning methods. We describe the inspiration for artificial neural networks and how the methods of deep learning are built. We define the activation function and its role in capturing nonlinear patterns in the input data. We explain the universal approximation theorem for understanding the power and limitation of these methods and describe the main topologies of artificial neural networks that play an important role in the successful implementation of these methods. We also describe loss functions (and their penalized versions) and give details about in which circumstances each of them should be used or preferred. In addition to the Ridge, Lasso, and Elastic Net regularization methods, we provide details of the dropout and the early stopping methods. Finally, we provide the backpropagation method and illustrate it with two simple artificial neural networks.Keywords
Funding Information
- Bill and Melinda Gates Foundation (INV-003439 BMGF/FCDO)
This publication has 25 references indexed in Scilit:
- Predicting the sequence specificities of DNA- and RNA-binding proteins by deep learningNature Biotechnology, 2015
- Deep learningNature, 2015
- Electricity price forecasting: A review of the state-of-the-art with a look into the futureInternational Journal of Forecasting, 2014
- Machine Learning Prediction of Cancer Cell Sensitivity to Drugs Based on Genomic and Chemical PropertiesPLOS ONE, 2013
- An Introduction to Statistical Learning2013
- Statistical aspects of neural networks1993
- Approximation capabilities of multilayer feedforward networksNeural Networks, 1991
- Approximation by superpositions of a sigmoidal functionMathematics of Control, Signals, and Systems, 1989
- Learning representations by back-propagating errorsNature, 1986
- A logical calculus of the ideas immanent in nervous activityBulletin of Mathematical Biology, 1943