EigenGame: PCA as a Nash Equilibrium
Authors:
- Ian GempIGIan Gemp
- Brian McWilliamsBMBrian McWilliams
- Claire VernadeCVClaire Vernade
- Thore GraepelTGThore Graepel
Preprint
Abstract
We present a novel view on principal component analysis (PCA) as a competitive game in which each approximate eigenvector is controlled by a player whose goal is to maximize their own utility function. We analyze the properties of this PCA game and the behavior of its gradient based updates. The resulting algorithm -- which combines elements from Oja's rule with a generalized Gram-Schmidt orthogonalization -- is naturally decentralized and hence parallelizable through message passing. We demonstrate the scalability of the algorithm with experiments on large image datasets and neural network activations. We discuss how this new view of PCA as a differentiable game can lead to further algorithmic developments and insights.Keywords
All Related Versions
This publication has 11 references indexed in Scilit:
- An Implicit Form of Krasulina's k-PCA Update without the Orthonormality ConstraintProceedings of the AAAI Conference on Artificial Intelligence, 2020
- Turning Big Data Into Tiny Data: Constant-Size Coresets for $k$-Means, PCA, and Projective ClusteringSIAM Journal on Computing, 2020
- Global rates of convergence for nonconvex optimization on manifoldsIMA Journal of Numerical Analysis, 2018
- Preserving privacy between features in distributed estimationStat, 2018
- Frequent Directions: Simple and Deterministic Matrix SketchingSIAM Journal on Computing, 2016
- The Complexity of Computing a Nash EquilibriumSIAM Journal on Computing, 2009
- Optimization Algorithms on Matrix Manifolds2008
- Nash and correlated equilibria: Some complexity considerationsGames and Economic Behavior, 1989
- Optimal unsupervised learning in a single-layer linear feedforward neural networkNeural Networks, 1989
- Simplified neuron model as a principal component analyzerJournal of Mathematical Biology, 1982