On Predicting the Optimal Number of Hidden Nodes
Authors:
- Alan J ThomasATAlan J Thomas
- Miltos PetridisMPMiltos Petridis
- Simon D WaltersSWSimon D Walters
- Saeed Malekshahi GheytassiSGSaeed Malekshahi Gheytassi
- Robert E MorganRobert E Morgan
- 1 December 2015
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 565-570
Abstract
Determining the optimal number of hidden nodes is the most challenging aspect of Artificial Neural Network (ANN) design. To date, there are still no reliable methods of determining this a priori, as it depends on so many domain-specific factors. Current methods which take these into account, such as exhaustive search, growing and pruning and evolutionary algorithms are not only inexact, but also extremely time consuming -- in some cases prohibitively so. A novel approach embodied in a system called Heurix is introduced. This rapidly predicts the optimal number of hidden nodes from a small number of sample topologies. It can be configured to favour speed (low complexity), accuracy, or a balance between the two. Single hidden layer feedforward networks (SLFNs) can be built twenty times faster, and with a generalisation error of as little as 0.4% greater than those found by exhaustive search.Keywords
This publication has 10 references indexed in Scilit:
- Empirical Analysis of Optimal Hidden Neurons in Neural Network Modeling for Stock Prediction2008
- Fast unit pruning algorithm for feedforward neural network designApplied Mathematics and Computation, 2008
- Learning capability and storage capacity of two-hidden-layer feedforward networksIEEE Transactions on Neural Networks, 2003
- Upper bounds on the number of hidden neurons in feedforward networks with arbitrary bounded nonlinear activation functionsIEEE Transactions on Neural Networks, 1998
- Constructive algorithms for structure learning in feedforward neural networks for regression problemsIEEE Transactions on Neural Networks, 1997
- Capabilities of a four-layered feedforward neural network: four layers versus threeIEEE Transactions on Neural Networks, 1997
- Pruning algorithms-a surveyIEEE Transactions on Neural Networks, 1993
- Kolmogorov's theorem and multilayer neural networksNeural Networks, 1992
- Multilayer feedforward networks are universal approximatorsNeural Networks, 1989
- On the structure of continuous functions of several variablesTransactions of the American Mathematical Society, 1965