Experimental Study on 164 Algorithms Available in Software Tools for Solving Standard Non-Linear Regression Problems
Open Access
- 5 August 2019
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 7, 108916-108939
Abstract
In the specialized literature, researchers can find a large number of proposals for solving regression problems that come from different research areas. However, researchers tend to use only proposals from the area in which they are experts. This paper analyses the performance of a large number of the available regression algorithms from some of the most known and widely used software tools in order to help non-expert users from other areas to properly solve their own regression problems and to help specialized researchers developing well-founded future proposals by properly comparing and identifying algorithms that will enable them to focus on significant further developments. To sum up, we have analyzed 164 algorithms that come from 14 main different families available in 6 software tools (Neural Networks, Support Vector Machines, Regression Trees, Rule-Based Methods, Stacking, Random Forests, Model trees, Generalized Linear Models, Nearest Neighbor methods, Partial Least Squares and Principal Component Regression, Multivariate Adaptive Regression Splines, Bagging, Boosting, and other methods) over 52 datasets. A new measure has also been proposed to show the goodness of each algorithm with respect to the others. Finally, a statistical analysis by non-parametric tests has been carried out over all the algorithms and on the best 30 algorithms, both with and without bagging. Results show that the algorithms from Random Forest, Model Tree and Support Vector Machine families get the best positions in the rankings obtained by the statistical tests when bagging is not considered. In addition, the use of bagging techniques significantly improves the performance of the algorithms without excessive increase in computational times.Keywords
Funding Information
- University of Cordoba (PPG2019-UCOSOCIAL-03)
- Spanish Ministry of Science, Innovation and Universities (TIN2015- 68454-R, TIN2017-89517-P)
This publication has 62 references indexed in Scilit:
- The WEKA data mining softwareACM SIGKDD Explorations Newsletter, 2009
- Unbiased Recursive Partitioning: A Conditional Inference FrameworkJournal of Computational and Graphical Statistics, 2006
- Spike and slab variable selection: Frequentist and Bayesian strategiesThe Annals of Statistics, 2005
- Stable and Efficient Multiple Smoothing Parameter Estimation for Generalized Additive ModelsJournal of the American Statistical Association, 2004
- Empirical evaluation of the improved Rprop learning algorithmsNeurocomputing, 2003
- Combining GP operators with SA search to evolve fuzzy rule based classifiersInformation Sciences, 2001
- The random subspace method for constructing decision forestsIEEE Transactions on Pattern Analysis and Machine Intelligence, 1998
- The Lack of A Priori Distinctions Between Learning AlgorithmsNeural Computation, 1996
- Multivariate Adaptive Regression SplinesThe Annals of Statistics, 1991
- Projection Pursuit RegressionJournal of the American Statistical Association, 1981