Faster SVM Training via Conjugate SMO
Preprint
Abstract
We propose an improved version of the SMO algorithm for training classification and regression SVMs, based on a Conjugate Descent procedure. This new approach only involves a modest increase on the computational cost of each iteration but, in turn, usually results in a substantial decrease in the number of iterations required to converge to a given precision. Besides, we prove convergence of the iterates of this new Conjugate SMO as well as a linear rate when the kernel matrix is positive definite. We have implemented Conjugate SMO within the LIBSVM library and show experimentally that it is faster for many hyper-parameter configurations, being often a better option than second order SMO when performing a grid-search for SVM tuning.Keywords
All Related Versions
This publication has 9 references indexed in Scilit:
- The Secrets of Machine Learning: Ten Things You Wish You Had Known Earlier to Be More Effective at Data Analysis2019
- Scaling Up Kernel SVM on Limited Resources: A Low-Rank Linearization ApproachIEEE Transactions on Neural Networks and Learning Systems, 2018
- Linear convergence rate for the MDM algorithm for the Nearest Point ProblemPattern Recognition, 2015
- Simple Proof of Convergence of the SMO Algorithm for Different SVM VariantsIEEE Transactions on Neural Networks and Learning Systems, 2012
- An improved GLMNET for l1-regularized logistic regression2011
- A Study on SMO-Type Decomposition Methods for Support Vector MachinesIEEE Transactions on Neural Networks, 2006
- General Polynomial Time Decomposition AlgorithmsLecture Notes in Computer Science, 2005
- Online Learning with KernelsIEEE Transactions on Signal Processing, 2004
- Asymptotic convergence of an SMO algorithm without any assumptionsIEEE Transactions on Neural Networks, 2002