Broad Minimax Probability Learning System and its Application in Regression Modeling
- 13 October 2022
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Systems, Man, and Cybernetics: Systems
- Vol. 53 (3) , 1945-1957
Abstract
Broad learning system (BLS), a novel incremental learning algorithm, has attracted more and more attention and application. BLS achieves a perfect balance between learning performance and modeling efficiency based on the mean square error (MSE) criterion. However, MSE assumes that a random error between an observed value and real value is a Gaussian distribution, which may be inconsistent to the real-world situation. Moreover, the distributional assumptions may cast doubt on the generalization and validity of the built BLS model. In this article, we propose a novel distribution-free BLS for regression analysis, called broad minimax probability learning system (BMPLS). Learning from the idea of minimax probability machine regression, BMPLS builds a regression model by maximizing the worst probability of the regression function being within the allowed error range. It adequately utilizes the mean and covariance information of the hidden-space feature without making any distributional assumptions of the random error. In order to further improve the learning performance of the model, a regularized BMPLS (RBMPLS) is proposed by applying elastic-net regularization term to BMPLS. The corresponding optimization algorithm is also designed for calculating the output weights of the model. Multiple experiments based on public datasets were carried out to demonstrate the feasibility of the proposed algorithms.Keywords
Funding Information
- Fundamental Research Funds for the Central Universities (2019XKQYMS64)
This publication has 35 references indexed in Scilit:
- Structural Minimax Probability MachineIEEE Transactions on Neural Networks and Learning Systems, 2016
- Robust Face Recognition via Minimum Error Entropy-Based Atomic RepresentationIEEE Transactions on Image Processing, 2015
- Coordinate descent algorithmsMathematical Programming, 2015
- Sparse Bayesian Extreme Learning Machine for Multi-classificationIEEE Transactions on Neural Networks and Learning Systems, 2013
- Regularization and Variable Selection Via the Elastic NetJournal of the Royal Statistical Society Series B: Statistical Methodology, 2005
- Ridge Regression: Biased Estimation for Nonorthogonal ProblemsTechnometrics, 2000
- Moment Problems and Semidefinite Optimization2000
- A rapid learning and dynamic stepwise updating algorithm for flat neural networks and the application to time-series predictionIEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), 1999
- The sample complexity of pattern classification with neural networks: the size of the weights is more important than the size of the networkIEEE Transactions on Information Theory, 1998
- Gradient-based learning applied to document recognitionProceedings of the IEEE, 1998