Generalized SMO Algorithm for SVM-Based Multitask Learning
Authors:
- 13 April 2012
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Neural Networks and Learning Systems
- Vol. 23 (6) , 997-1003
Abstract
Exploiting additional information to improve traditional inductive learning is an active research area in machine learning. In many supervised-learning applications, training data can be naturally separated into several groups, and incorporating this group information into learning may improve generalization. Recently, Vapnik proposed a general approach to formalizing such problems, known as “learning with structured data” and its support vector machine (SVM) based optimization formulation called SVM+. Liang and Cherkassky showed the connection between SVM+ and multitask learning (MTL) approaches in machine learning, and proposed an SVM-based formulation for MTL called SVM+MTL for classification. Training the SVM+MTL classifier requires the solution of a large quadratic programming optimization problem which scales as O(n 3 ) with sample size n. So there is a need to develop computationally efficient algorithms for implementing SVM+MTL. This brief generalizes Platt's sequential minimal optimization (SMO) algorithm to the SVM+MTL setting. Empirical results show that, for typical SVM+MTL problems, the proposed generalized SMO achieves over 100 times speed-up, in comparison with general-purpose optimization routines.Keywords
This publication has 6 references indexed in Scilit:
- Predictive learning with structured (grouped) dataNeural Networks, 2009
- Uncovering shared structures in multiclass classification2007
- Constructing informative priors using transfer learning2006
- Regularized multi--task learning2004
- Learning to learn with the informative vector machine2004
- Support-vector networksMachine Learning, 1995