An Ensemble Learning Approach Based on Random Sparse Quadratic Twin Parametric-Margin SVMs
Authors:
Preprint
- 29 June 2026
- preprint
- Published by Elsevier in SSRN Electronic Journal
Abstract
Ensemble learning methods based on support vector machines (SVMs) have shown promising generalization ability in nonlinear classification tasks. However, the heteroscedastic noise in real-world datasets, simple classifier of base SVMs learners, kernel selection and kernel parameter tuning limit the classifying performance of SVMs with kernels and their related ensemble learning methods. To address these issues, we first propose a least squares kernel-free quadratic twin parametric-margin SVM model (LSQTPSVM) as the base learner within an ensemble framework. The proposed model introduces a parametric-margin term to handle heteroscedastic noise and employs two quadratic surfaces to capture complex data distributions to improve its robustness. Then we introduce a novel diversification strategy that randomly alters the parameters associated with linear, interaction and quadratic terms of two quadratic surfaces yielded by base learners, to develop an ensemble learning method based on the LSQTPSVM model. Comprehensive numerical results on synthetic and public benchmark datasets indicate that the proposed ensemble classification method outperforms commonly-used classification models and ensemble methods in terms of classification accuracy. Finally, we apply the ensemble learning method to credit risk assessment for validating its effectiveness in handling such real-world problems.Keywords
This publication has 22 references indexed in Scilit:
- Inherently interpretable machine learning for credit scoring: Optimal classification tree with hyperplane splitsEuropean Journal of Operational Research, 2024
- Sparse L1-norm quadratic surface support vector machine with Universum dataSoft Computing, 2023
- A novel robust nonparallel support vector classifier based on one optimization problemNeural Computing and Applications, 2022
- Comprehensive review on twin support vector machinesAnnals of Operations Research, 2022
- A comparative analysis of gradient boosting algorithmsArtificial Intelligence Review, 2020
- Unsupervised quadratic surface support vector machine with application to credit risk assessmentEuropean Journal of Operational Research, 2020
- Quadratic Kernel-Free Least Square Twin Support Vector Machine for Binary Classification ProblemsJournal of the Operations Research Society of China, 2019
- A support vector machine-based ensemble algorithm for breast cancer diagnosisEuropean Journal of Operational Research, 2018
- Performance Degradation in Boosting2001
- A Decision-Theoretic Generalization of On-Line Learning and an Application to BoostingJournal of Computer and System Sciences, 1997