Online Adaptive Estimation of Sparse Signals: Where RLS Meets the $\ell_1$-Norm
Top Cited Papers
- 29 March 2010
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Signal Processing
- Vol. 58 (7) , 3436-3447
Abstract
Using the ℓ1-norm to regularize the least-squares criterion, the batch least-absolute shrinkage and selection operator (Lasso) has well-documented merits for estimating sparse signals of interest emerging in various applications where observations adhere to parsimonious linear regression models. To cope with high complexity, increasing memory requirements, and lack of tracking capability that batch Lasso estimators face when processing observations sequentially, the present paper develops a novel time-weighted Lasso (TWL) approach. Performance analysis reveals that TWL cannot estimate consistently the desired signal support without compromising rate of convergence. This motivates the development of a time- and norm-weighted Lasso (TNWL) scheme with ℓ1-norm weights obtained from the recursive least-squares (RLS) algorithm. The resultant algorithm consistently estimates the support of sparse signals without reducing the convergence rate. To cope with sparsity-aware recursive real-time processing, novel adaptive algorithms are also developed to enable online coordinate descent solvers of TWL and TNWL that provably converge to the true sparse signal in the time-invariant case. Simulated tests compare competing alternatives and corroborate the performance of the novel algorithms in estimating time-invariant signals, and tracking time-varying signals under sparsity constraints.Keywords
This publication has 25 references indexed in Scilit:
- One-step sparse estimates in nonconcave penalized likelihood modelsThe Annals of Statistics, 2008
- Low-Complexity RLS Algorithms Using Dichotomous Coordinate Descent IterationsIEEE Transactions on Signal Processing, 2008
- Coordinate descent algorithms for lasso penalized regressionThe Annals of Applied Statistics, 2008
- Compressed sensing with sequential observations2008
- Pathwise coordinate optimizationThe Annals of Applied Statistics, 2007
- Stable signal recovery from incomplete and inaccurate measurementsCommunications on Pure and Applied Mathematics, 2006
- Partial update LMS algorithmsIEEE Transactions on Signal Processing, 2005
- Variable Selection via Nonconcave Penalized Likelihood and its Oracle PropertiesJournal of the American Statistical Association, 2001
- Asymptotics for lasso-type estimatorsThe Annals of Statistics, 2000
- The Risk Inflation Criterion for Multiple RegressionThe Annals of Statistics, 1994