AN IMPROVEMENT OF EXTREME LEARNING MACHINE FOR COMPACT SINGLE-HIDDEN-LAYER FEEDFORWARD NEURAL NETWORKS
- 1 October 2008
- journal article
- research article
- Published by World Scientific Pub Co Pte Ltd in International Journal of Neural Systems
- Vol. 18 (05) , 433-441
Abstract
Recently, a novel learning algorithm called extreme learning machine (ELM) was proposed for efficiently training single-hidden-layer feedforward neural networks (SLFNs). It was much faster than the traditional gradient-descent-based learning algorithms due to the analytical determination of output weights with the random choice of input weights and hidden layer biases. However, this algorithm often requires a large number of hidden units and thus slowly responds to new observations. Evolutionary extreme learning machine (E-ELM) was proposed to overcome this problem; it used the differential evolution algorithm to select the input weights and hidden layer biases. However, this algorithm required much time for searching optimal parameters with iterative processes and was not suitable for data sets with a large number of input features. In this paper, a new approach for training SLFNs is proposed, in which the input weights and biases of hidden units are determined based on a fast regularized least-squares scheme. Experimental results for many real applications with both small and large number of input features show that our proposed approach can achieve good generalization performance with much more compact networks and extremely high speed for both learning and testing.Keywords
This publication has 16 references indexed in Scilit:
- Convex incremental extreme learning machineNeurocomputing, 2007
- Universal Approximation using Incremental Constructive Feedforward Networks with Random Hidden NodesIEEE Transactions on Neural Networks, 2006
- Evolutionary extreme learning machinePattern Recognition, 2005
- Efficient tuning of SVM hyperparameters using radius/margin bound and iterative algorithmsIEEE Transactions on Neural Networks, 2002
- A comparison of methods for multiclass support vector machinesIEEE Transactions on Neural Networks, 2002
- Gene expression profiling predicts clinical outcome of breast cancerNature, 2002
- Feedforward networks training speed enhancement by optimal initialization of the synaptic coefficientsIEEE Transactions on Neural Networks, 2001
- Classification ability of single hidden layer feedforward neural networksIEEE Transactions on Neural Networks, 2000
- The sample complexity of pattern classification with neural networks: the size of the weights is more important than the size of the networkIEEE Transactions on Information Theory, 1998
- Efficient BackProp1998