Trace Norm Regularization: Reformulations, Algorithms, and Multi-Task Learning
Authors:
- Ting Kei PongTPTing Kei Pong
- Paul TsengPTPaul Tseng
- Shuiwang JiSJShuiwang Ji
- Jieping YeJYJieping Ye
- 1 January 2010
- journal article
- Published by Society for Industrial & Applied Mathematics (SIAM) in SIAM Journal on Optimization
- Vol. 20 (6) , 3465-3489
Abstract
We consider a recently proposed optimization formulation of multi-task learning based on trace norm regularized least squares. While this problem may be formulated as a semidefinite program (SDP), its size is beyond general SDP solvers. Previous solution approaches apply proximal gradient methods to solve the primal problem. We derive new primal and dual reformulations of this problem, including a reduced dual formulation that involves minimizing a convex quadratic function over an operator-norm ball in matrix space. This reduced dual problem may be solved by gradient-projection methods, with each projection involving a singular value decomposition. The dual approach is compared with existing approaches and its practical effectiveness is illustrated on simulations and an application to gene expression pattern analysis.Keywords
This publication has 30 references indexed in Scilit:
- A Singular Value Thresholding Algorithm for Matrix CompletionSIAM Journal on Optimization, 2010
- A bag-of-words approach for Drosophila gene expression pattern annotationBMC Bioinformatics, 2009
- Exact Matrix Completion via Convex OptimizationFoundations of Computational Mathematics, 2009
- A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse ProblemsSIAM Journal on Imaging Sciences, 2009
- Mapping the gene expression universeCurrent Opinion in Genetics & Development, 2008
- Automated annotation of Drosophila gene expression patterns using a controlled vocabularyBioinformatics, 2008
- Image retrievalACM Computing Surveys, 2008
- Convex multi-task feature learningMachine Learning, 2008
- A Model of Inductive Bias LearningJournal of Artificial Intelligence Research, 2000
- Multitask LearningMachine Learning, 1997