Estimating Multiple Precision Matrices with Cluster Fusion Regularization
Preprint
Abstract
We propose a penalized likelihood framework for estimating multiple precision matrices from different classes. Most existing methods either incorporate no information on relationships between the precision matrices, or require this information be known a priori. The framework proposed in this article allows for simultaneous estimation of the precision matrices and relationships between the precision matrices, jointly. Sparse and non-sparse estimators are proposed, both of which require solving a non-convex optimization problem. To compute our proposed estimators, we use an iterative algorithm which alternates between a convex optimization problem solved by blockwise coordinate descent and a k-means clustering problem. Blockwise updates for computing the sparse estimator require solving an elastic net penalized precision matrix estimation problem, which we solve using a proximal gradient descent algorithm. We prove that this subalgorithm has a linear rate of convergence. In simulation studies and two real data applications, we show that our method can outperform competitors that ignore relevant relationships between precision matrices and performs similarly to methods which use prior information often uknown in practice.Keywords
All Related Versions
This publication has 16 references indexed in Scilit:
- An overview of the estimation of large covariance and precision matricesThe Econometrics Journal, 2016
- Ridge Fusion in Statistical LearningJournal of Computational and Graphical Statistics, 2015
- Structural Pursuit Over Multiple Undirected GraphsJournal of the American Statistical Association, 2014
- The Cluster Elastic Net for High-Dimensional Regression With Unknown Variable GroupingTechnometrics, 2014
- The Joint Graphical Lasso for Inverse Covariance Estimation Across Multiple ClassesJournal of the Royal Statistical Society Series B: Statistical Methodology, 2013
- Likelihood-Based Selection and Sharp Parameter EstimationJournal of the American Statistical Association, 2012
- Joint estimation of multiple graphical modelsBiometrika, 2011
- Covariance-Regularized Regression and Classification for high Dimensional ProblemsJournal of the Royal Statistical Society Series B: Statistical Methodology, 2009
- Model selection and estimation in the Gaussian graphical modelBiometrika, 2007
- Genetic K-means algorithmIEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), 1999