Consistency of the group Lasso and multiple kernel learning
Author:
Preprint
Abstract
We consider the least-square regression problem with regularization by a block 1-norm, i.e., a sum of Euclidean norms over spaces of dimensions larger than one. This problem, referred to as the group Lasso, extends the usual regularization by the 1-norm where all spaces have dimension one, where it is commonly referred to as the Lasso. In this paper, we study the asymptotic model consistency of the group Lasso. We derive necessary and sufficient conditions for the consistency of group Lasso under practical assumptions, such as model misspecification. When the linear predictors and Euclidean norms are replaced by functions and reproducing kernel Hilbert norms, the problem is usually referred to as multiple kernel learning and is commonly used for learning from heterogeneous data sources and for non linear variable selection. Using tools from functional analysis, and in particular covariance operators, we extend the consistency results to this infinite dimensional case and also propose an adaptive scheme to obtain a consistent model estimate, even when the necessary condition required for the non adaptive scheme is not satisfied.Keywords
All Related Versions
This publication has 14 references indexed in Scilit:
- The Group Lasso for Logistic RegressionJournal of the Royal Statistical Society Series B: Statistical Methodology, 2008
- On the Non-Negative Garrotte EstimatorJournal of the Royal Statistical Society Series B: Statistical Methodology, 2007
- Multi-kernel regularized classifiersJournal of Complexity, 2007
- The Adaptive Lasso and Its Oracle PropertiesJournal of the American Statistical Association, 2006
- Model Selection and Estimation in Regression with Grouped VariablesJournal of the Royal Statistical Society Series B: Statistical Methodology, 2005
- Subband correlation and robust speech recognitionIEEE Transactions on Speech and Audio Processing, 2005
- A statistical framework for genomic data fusionBioinformatics, 2004
- Reproducing Kernel Hilbert Spaces in Probability and Statistics2004
- Applications of second-order cone programmingLinear Algebra and its Applications, 1998
- Regression Shrinkage and Selection Via the LassoJournal of the Royal Statistical Society Series B: Statistical Methodology, 1996