Online dictionary learning from big data using accelerated stochastic approximation algorithms
- 1 May 2014
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- No. 15206149,p. 16-20
Abstract
Applications involving large-scale dictionary learning tasks motivate well online optimization algorithms for generally non-convex and non-smooth problems. In this big data context, the present paper develops an online learning framework by jointly leveraging the stochastic approximation paradigm with first-order acceleration schemes. The generally non-convex objective evaluated online at the resultant iterates enjoys quadratic rate of convergence. The generality of the novel approach is demonstrated in two online learning applications: (i) Online linear regression using the total least-squares approach; and, (ii) a semi-supervised dictionary learning approach to network-wide link load tracking and imputation of real data with missing entries. In both cases, numerical tests highlight the potential of the proposed online framework for big data network analytics.Keywords
This publication has 23 references indexed in Scilit:
- Decentralized Sparsity-Regularized Rank Minimization: Algorithms and ApplicationsIEEE Transactions on Signal Processing, 2013
- A Block Coordinate Descent Method for Regularized Multiconvex Optimization with Applications to Nonnegative Tensor Factorization and CompletionSIAM Journal on Imaging Sciences, 2013
- A Unified Convergence Analysis of Block Successive Minimization Methods for Nonsmooth OptimizationSIAM Journal on Optimization, 2013
- Online Learning and Online Convex OptimizationFoundations and Trends in Machine Learning, 2012
- Stochastic Coordinate Descent Methods for Regularized Smooth and Nonsmooth Losses2012
- Over-relaxation of the fast iterative shrinkage-thresholding algorithm with variable stepsizeInverse Problems, 2011
- Distributed Optimization and Statistical Learning via the Alternating Direction Method of MultipliersFoundations and Trends in Machine Learning, 2011
- Fast Gradient-Based Algorithms for Constrained Total Variation Image Denoising and Deblurring ProblemsIEEE Transactions on Image Processing, 2009
- Sparse Reconstruction by Separable ApproximationIEEE Transactions on Signal Processing, 2009
- A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse ProblemsSIAM Journal on Imaging Sciences, 2009