ERDeR: The Combination of Statistical Shrinkage Methods and Ensemble Approaches to Improve the Performance of Deep Regression
Open Access
- 21 February 2024
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 12, 33361-33383
Abstract
Ensembling is a powerful technique to obtain the most accurate results. In some cases, the large number of learners in ensemble learning mostly increases both computational load during the test phase and error rate. To solve this problem, in this paper we propose an Ensemble of Reduced Deep Regression (ERDeR) model, which is a combination of Deep Regressions (DRs), shrinkage methods, and ensemble approaches. The framework of the proposed model contains three phases. The first phase includes base regressions in which parallel DRs are used as learners. The role of these DRs is to extract features of input data and make prediction. In the second phase, to automatically reduce and select the most suitable DRs, shrinkage methods such as Least Absolute Shrinkage and Selection Operator (LASSO) and Elastic Net (EN) are employed. These models are compared with the non-shrinkage model. The last phase is ensemble phase, which consists of three different ensemble methods namely Multi-Layer Perceptron (MLP), Weighted Average (WA), and Simple Average (SA). These ensemble methods are used to aggregate the remaining learners from previous steps. Finally, the proposed model is applied to Monte Carlo simulation data and three real datasets including Boston House Price, Real Estate Valuation and Gold Price per Ounce. The results show that after applying the shrinkage methods the error rate is significantly reduced and the model accuracy is increased. Accordingly, the results of combining shrinkage methods and ensemble approaches not only decreased the computational load during test phase, but also increased the model accuracy.Keywords
Funding Information
- National Research Foundation of Korea
- Korea government (RS-2023-00239877)
This publication has 82 references indexed in Scilit:
- REDS: Random ensemble deep spatial predictionEnvironmetrics, 2022
- Remote sensing‐based retrieval of soil moisture content using stacking ensemble learning modelsLand Degradation & Development, 2022
- Stacking Ensemble Learning-Based Wireless Sensor Network Deployment Parameter EstimationArabian Journal for Science and Engineering, 2022
- Automatic personality prediction: an enhanced method using ensemble modelingNeural Computing and Applications, 2022
- Enhanced feature selection method based on regularization and kernel trick for 5G applications and beyondAlexandria Engineering Journal, 2022
- A structural damage detection algorithm based on discrete wavelet transform and ensemble pattern recognition modelsJournal of Civil Structural Health Monitoring, 2022
- CWI: A multimodal deep learning approach for named entity recognition from social media using character, word and image featuresNeural Computing and Applications, 2021
- Estate price prediction system based on temporal and spatial features and lightweight deep learning modelApplied Intelligence, 2021
- Model Selection in Committees of Evolved Convolutional Neural Networks Using Genetic Algorithms2018
- Deep Regression Tracking with Shrinkage Loss2018