A Filter-SQP strategy for training Support Vector Machine models
Open Access
- 31 July 2023
- journal article
- research article
- Published by Instituto Federal de Educacao - Ciencia e Tecnologia do Rio Grande do Sul in REMAT: Revista Eletrônica da Matemática
- Vol. 9 (2) , e3004
Abstract
This paper introduces a filtering strategy for addressing optimization problems arising in binary Support Vector Machine classification. The training optimization problem aims to solve the dual formulation which involves a quadratic objective function subjected to a linear and box constraints. Our approach employs a Filter algorithm with Sequential Quadratic Programming iterations that minimize the quadratic Lagrangian approximations. Notably, we utilize the exact Hessian matrix in our numerical experiments to seek the desired classification function. Moreover, we present a Filter algorithm combined with the Augmented Lagrangian method aiming to accelerate the algorithm convergence. To substantiate our method's effectiveness, we conduct numerical experiments through MATLAB, comparing outcomes with alternative methodologies detailed in existing literature. Numerical experiments shows that the Filter--SQP combined with Augmented Lagrangian method is competitive and efficient method compared with an interior-point based solver and LIBSVM software in relation of classification metrics and CPU-time.Keywords
This publication has 27 references indexed in Scilit:
- Nonlinear optimization and support vector machines4OR, 2018
- Problem formulations and solvers in linear SVM: a reviewArtificial Intelligence Review, 2018
- Newton-Type Methods for Optimization and Variational Problems2014
- Pegasos: primal estimated sub-gradient solver for SVMMathematical Programming, 2010
- Exploiting separability in large-scale linear support vector machine trainingComputational Optimization and Applications, 2009
- The Elements of Statistical Learning2009
- Numerical Optimization2006
- Benchmarking optimization software with performance profilesMathematical Programming, 2002
- Nonlinear programming without a penalty functionMathematical Programming, 2002
- Support-vector networksMachine Learning, 1995