On Training Efficiency and Computational Costs of a Feed Forward Neural Network: A Review
Open Access
- 31 August 2015
- journal article
- review article
- Published by Hindawi Limited in Computational Intelligence and Neuroscience
- Vol. 2015, 1-13
Abstract
A comprehensive review on the problem of choosing a suitable activation function for the hidden layer of a feed forward neural network has been widely investigated. Since the nonlinear component of a neural network is the main contributor to the network mapping capabilities, the different choices that may lead to enhanced performances, in terms of training, generalization, or computational costs, are analyzed, both in general-purpose and in embedded computing environments. Finally, a strategy to convert a network configuration between different activation functions without altering the network mapping capabilities will be presented.Keywords
This publication has 35 references indexed in Scilit:
- Chaotic Time Series Prediction Based on a Novel Robust Echo State NetworkIEEE Transactions on Neural Networks and Learning Systems, 2012
- Minimum Complexity Echo State NetworkIEEE Transactions on Neural Networks, 2010
- Echo state networkScholarpedia, 2007
- Neural networks with a continuous squashing function in the output are universal approximatorsNeural Networks, 2000
- Learning long-term dependencies in NARX recurrent neural networksIEEE Transactions on Neural Networks, 1996
- Multilayer feedforward networks with a nonpolynomial activation function can approximate any functionNeural Networks, 1993
- Approximation capabilities of multilayer feedforward networksNeural Networks, 1991
- Approximation by superpositions of a sigmoidal functionMathematics of Control, Signals, and Systems, 1989
- On the approximate realization of continuous mappings by neural networksNeural Networks, 1989
- Multilayer feedforward networks are universal approximatorsNeural Networks, 1989