Extreme Learning Machine for Regression and Multiclass Classification
Authors:
Top Cited Papers
- 6 October 2011
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics)
- Vol. 42 (2) , 513-529
Abstract
Due to the simplicity of their implementations, least square support vector machine (LS-SVM) and proximal support vector machine (PSVM) have been widely used in binary classification applications. The conventional LS-SVM and PSVM cannot be used in regression and multiclass classification applications directly, although variants of LS-SVM and PSVM have been proposed to handle such cases. This paper shows that both LS-SVM and PSVM can be simplified further and a unified learning framework of LS-SVM, PSVM, and other regularization algorithms referred to extreme learning machine (ELM) can be built. ELM works for the “generalized” single-hidden-layer feedforward networks (SLFNs), but the hidden layer (or called feature mapping) in ELM need not be tuned. Such SLFNs include but are not limited to SVM, polynomial network, and the conventional feedforward neural networks. This paper shows the following: 1) ELM provides a unified learning platform with a widespread type of feature mappings and can be applied in regression and multiclass classification applications directly; 2) from the optimization method point of view, ELM has milder optimization constraints compared to LS-SVM and PSVM; 3) in theory, compared to ELM, LS-SVM and PSVM achieve suboptimal solutions and require higher computational complexity; and 4) in theory, ELM can approximate any target continuous function and classify any disjoint regions. As verified by the simulation results, ELM tends to have better scalability and achieve similar (for regression and binary class cases) or much better (for multiclass cases) generalization performance at much faster learning speed (up to thousands times) than traditional SVM and LS-SVM.Keywords
This publication has 42 references indexed in Scilit:
- Optimization method based extreme learning machine for classificationNeurocomputing, 2010
- Error Minimized Extreme Learning Machine With Growth of Hidden Nodes and Incremental LearningIEEE Transactions on Neural Networks, 2009
- Online Sequential Fuzzy Extreme Learning Machine for Function Approximation and Classification ProblemsIEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), 2009
- Incremental extreme learning machine with fully complex hidden nodesNeurocomputing, 2007
- Classification ability of single hidden layer feedforward neural networksIEEE Transactions on Neural Networks, 2000
- Training multilayer perceptron classifiers based on a modified support vector methodIEEE Transactions on Neural Networks, 1999
- Least Squares Support Vector Machine ClassifiersNeural Processing Letters, 1999
- The sample complexity of pattern classification with neural networks: the size of the weights is more important than the size of the networkIEEE Transactions on Information Theory, 1998
- Support-vector networksMachine Learning, 1995
- Ridge Regression: Biased Estimation for Nonorthogonal ProblemsTechnometrics, 1970