Robust Extreme Learning Machine Using New Activation and Loss Functions Based on M-Estimation for Regression and Classification
Open Access
- 15 October 2022
- journal article
- research article
- Published by Hindawi Limited in Scientific Programming
- Vol. 2022, 1-10
Abstract
This paper provides an analysis of the combining effect of novel activation function and loss function based on M-estimation in application to extreme learning machine (ELM), a feed-forward neural network. Due to the computational efficiency and classification/prediction accuracy of ELM and its variants, they have been widely exploited in the development of new technologies and applications. However, in real applications, the performance of classical ELMs deteriorates in the presence of outliers, thus, negatively impacting the precision and accuracy of the system. To further enhance the performance of ELM and its variants, we proposed novel activation functions based on the psi function of M and redescend the M-estimation method along with the smooth l 2-norm weight-loss functions to reduce the negative impact of the outliers. The proposed psi functions of several M and redescending M-estimation methods are more flexible to make more distinct features space. For the first time, the idea of the psi function as an activation function in the neural network is introduced in the literature to ensure accurate prediction. In addition, new robust l 2 norm-loss functions based on M and redescending M-estimation are proposed to deal with outliers efficiently in ELM. To evaluate the performance of the proposed methodology against other state-of-the-art techniques, experiments have been performed in diverse environments, which show promising improvements in application to regression and classification problems.Keywords
This publication has 28 references indexed in Scilit:
- What are Extreme Learning Machines? Filling the Gap Between Frank Rosenblatt’s Dream and John von Neumann’s PuzzleCognitive Computation, 2015
- Outlier-robust extreme learning machine for regression problemsNeurocomputing, 2015
- Trends in extreme learning machines: A reviewNeural Networks, 2015
- Robust activation function and its application: Semi-supervised kernel extreme learning methodNeurocomputing, 2014
- Extreme learning machine and its applicationsNeural Computing and Applications, 2013
- Robust extreme learning machineNeurocomputing, 2013
- Universal Approximation of Extreme Learning Machine With Adaptive Growth of Hidden NodesIEEE Transactions on Neural Networks and Learning Systems, 2012
- Extreme learning machines: a surveyInternational Journal of Machine Learning and Cybernetics, 2011
- Comparison of new activation functions in neural network for forecasting financial time seriesNeural Computing and Applications, 2010
- Insha’s Redescending M-estimator for Robust Regression: A Comparative StudyPakistan Journal of Statistics and Operation Research, 2006