K-Deep Simplex: Manifold Learning via Local Dictionaries
- 12 October 2023
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Signal Processing
- Vol. 71 (1053587X) , 3741-3754
Abstract
We propose K- Deep Simplex (KDS) which, given a set of data points, learns a dictionary comprising synthetic landmarks, along with representation coefficients supported on a simplex. KDS employs a local weighted ℓ 1 penalty that encourages each data point to represent itself as a convex combination of nearby landmarks. We solve the proposed optimization program using alternating minimization and design an efficient, interpretable autoencoder using algorithm unrolling. We theoretically analyse the proposed program by relating the weighted ℓ 1 penalty in KDS to a weighted ℓ 0 program. Assuming that the data are generated from a Delaunay triangulation, we prove the equivalence of the weighted ℓ 1 and weighted ℓ 0 programs. We further show the stability of the representation coefficients under mild geometrical assumptions. If the representation coefficients are fixed, we prove that the sub-problem of minimizing over the dictionary yields a unique solution. Further, we show that low-dimensional representations can be efficiently obtained from the covariance of the coefficient matrix. Experiments show that the algorithm is highly efficient and performs competitively on synthetic and real data sets.Keywords
Funding Information
- NSF (DMS-2208392, DMS-1912737, DMS-1924513, DMS-2309519, DMS-2318894, DMS-2134157, PHY-2019786)
This publication has 54 references indexed in Scilit:
- Fast projection onto the simplex and the $\pmb {l}_\mathbf {1}$ l 1 ballMathematical Programming, 2015
- Network Lasso2015
- Smooth Representation Clustering2014
- Multi-scale geometric methods for data sets II: Geometric Multi-Resolution AnalysisApplied and Computational Harmonic Analysis, 2012
- From Sparse Solutions of Systems of Equations to Sparse Modeling of Signals and ImagesSIAM Review, 2009
- A tutorial on spectral clusteringStatistics and Computing, 2007
- $rm K$-SVD: An Algorithm for Designing Overcomplete Dictionaries for Sparse RepresentationIEEE Transactions on Signal Processing, 2006
- Diffusion mapsApplied and Computational Harmonic Analysis, 2006
- Laplacian Eigenmaps for Dimensionality Reduction and Data RepresentationNeural Computation, 2003
- Emergence of simple-cell receptive field properties by learning a sparse code for natural imagesNature, 1996