Generative Principal Component Analysis
Preprint
Abstract
In this paper, we study the problem of principal component analysis with generative modeling assumptions, adopting a general model for the observed matrix that encompasses notable special cases, including spiked matrix recovery and phase retrieval. The key assumption is that the underlying signal lies near the range of an $L$-Lipschitz continuous generative model with bounded $k$-dimensional inputs. We propose a quadratic estimator, and show that it enjoys a statistical rate of order $\sqrt{\frac{k\log L}{m}}$, where $m$ is the number of samples. We also provide a near-matching algorithm-independent lower bound. Moreover, we provide a variant of the classic power method, which projects the calculated data onto the range of the generative model during each iteration. We show that under suitable conditions, this method converges exponentially fast to a point achieving the above-mentioned statistical rate. We perform experiments on various image datasets for spiked matrix and phase retrieval models, and illustrate performance gains of our method to the classic power method and the truncated power method devised for sparse principal component analysis.
Keywords
All Related Versions
This publication has 24 references indexed in Scilit:
- Robust 1-bit Compressive Sensing with Partial Gaussian Circulant Matrices and Generative Priors2021
- Compressed Sensing-Based Robust Phase Retrieval via Deep Generative PriorsIEEE Sensors Journal, 2020
- Solving Inverse Problems via Auto-EncodersIEEE Journal on Selected Areas in Information Theory, 2020
- Deep Learning Techniques for Inverse Problems in ImagingIEEE Journal on Selected Areas in Information Theory, 2020
- Information-Theoretic Lower Bounds for Compressive Sensing With Generative ModelsIEEE Journal on Selected Areas in Information Theory, 2020
- The Informativeness of $k$ -Means for Learning Mixture ModelsIEEE Transactions on Information Theory, 2019
- The Projected Power Method: An Efficient Algorithm for Joint Alignment from Pairwise DifferencesCommunications on Pure and Applied Mathematics, 2018
- Phase Retrieval via Wirtinger Flow: Theory and AlgorithmsIEEE Transactions on Information Theory, 2015
- Introduction to the non-asymptotic analysis of random matrices2012
- Singular value decomposition for genome-wide expression data processing and modelingProceedings of the National Academy of Sciences of the United States of America, 2000