Online Orthogonal Dictionary Learning Based on Frank–Wolfe Method
- 8 December 2021
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Neural Networks and Learning Systems
- Vol. 34 (9) , 5774-5788
Abstract
Dictionary learning is a widely used unsupervised learning method in signal processing and machine learning. Most existing works on dictionary learning adopt an off-line approach, and there are two main off-line ways of conducting it. One is to alternately optimize both the dictionary and the sparse code, while the other is to optimize the dictionary by restricting it over the orthogonal group. The latter, called orthogonal dictionary learning (ODL), has a lower implementation complexity and, hence, is more favorable for low-cost devices. However, existing schemes for ODL only work with batch data and cannot be implemented online, making them inapplicable for real-time applications. This article, thus, proposes a novel online orthogonal dictionary scheme to dynamically learn the dictionary from streaming data, without storing the historical data. The proposed scheme includes a novel problem formulation and an efficient online algorithm design with convergence analysis. In the problem formulation, we relax the orthogonal constraint to enable an efficient online algorithm. We then propose the design of a new Frank-Wolfe-based online algorithm with a convergence rate of O(łn t/t1/4). The convergence rate in terms of key system parameters is also derived. Experiments with synthetic data and real-world Internet of things (IoT) sensor readings demonstrate the effectiveness and efficiency of the proposed online ODL scheme.Keywords
All Related Versions
This publication has 25 references indexed in Scilit:
- Beamforming matrix quantization with variable feedback rateEURASIP Journal on Wireless Communications and Networking, 2012
- Projection-like Retractions on Matrix ManifoldsSIAM Journal on Optimization, 2012
- Sentiment Knowledge Discovery in Twitter Streaming Data2010
- Sparse principal component analysis via regularized low rank matrix approximationJournal of Multivariate Analysis, 2008
- $rm K$-SVD: An Algorithm for Designing Overcomplete Dictionaries for Sparse RepresentationIEEE Transactions on Signal Processing, 2006
- Continuous queries over data streamsACM SIGMOD Record, 2001
- Testing the equality of prediction mean squared errorsInternational Journal of Forecasting, 1997
- A parallel algorithm for computing the polar decompositionParallel Computing, 1994
- An algorithm for quadratic programmingNaval Research Logistics Quarterly, 1956
- On classes of summable functions and their Fourier SeriesProceedings of the Royal Society of London. Series A, Containing Papers of a Mathematical and Physical Character, 1912