Cross Validation Through Two-Dimensional Solution Surface for Cost-Sensitive SVM
Authors:
Open Access
- 8 June 2016
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Pattern Analysis and Machine Intelligence
- Vol. 39 (6) , 1103-1121
Abstract
Model selection plays an important role in cost-sensitive SVM (CS-SVM). It has been proven that the global minimum cross validation (CV) error can be efficiently computed based on the solution path for one parameter learning problems. However, it is a challenge to obtain the global minimum CV error for CS-SVM based on one-dimensional solution path and traditional grid search, because CS-SVM is with two regularization parameters. In this paper, we propose a solution and error surfaces based CV approach (CV-SES). More specifically, we first compute a two-dimensional solution surface for CS-SVM based on a bi-parameter space partition algorithm, which can fit solutions of CS-SVM for all values of both regularization parameters. Then, we compute a two-dimensional validation error surface for each CV fold, which can fit validation errors of CS-SVM for all values of both regularization parameters. Finally, we obtain the CV error surface by superposing K validation error surfaces, which can find the global minimum CV error of CSSVM. Experiments are conducted on seven datasets for cost sensitive learning and on four datasets for imbalanced learning. Experimental results not only show that our proposed CV-SES has a better generalization ability than CS-SVM with various hybrids between grid search and solution path methods, and than recent proposed cost-sensitive hinge loss SVM with three-dimensional grid search, but also show that CV-SES uses less running time.Keywords
Funding Information
- Priority Academic Program Development (PAPD)
- U.S. National Science Foundation (IIS-1115417)
- National Natural Science Foundation of China (61232016, 61573191, 61573191)
This publication has 20 references indexed in Scilit:
- Cost-Sensitive Learning via Priority Sampling to Improve the Return on Marketing and CRM InvestmentJournal of Management Information Systems, 2012
- LIBSVMACM Transactions on Intelligent Systems and Technology, 2011
- Determination of Global Minima of Some Common Validation Functions in Support Vector MachineIEEE Transactions on Neural Networks, 2011
- Nested Support Vector MachinesIEEE Transactions on Signal Processing, 2009
- Learning from Imbalanced DataIEEE Transactions on Knowledge and Data Engineering, 2009
- Nonparametric Conditional Density Estimation Using Piecewise-Linear Solution Path of Kernel Quantile RegressionNeural Computation, 2009
- A New Solution Path Algorithm in Support Vector RegressionIEEE Transactions on Neural Networks, 2008
- Piecewise linear regularized solution pathsThe Annals of Statistics, 2007
- Efficient Computation and Model Selection for the Support Vector RegressionNeural Computation, 2007
- New Support Vector AlgorithmsNeural Computation, 2000