Robust reduced-rank regression
Authors:
Open Access
- 12 July 2017
- journal article
- research article
- Published by Oxford University Press (OUP) in Biometrika
- Vol. 104 (3) , 633-647
Abstract
In high-dimensional multivariate regression problems, enforcing low rank in the coefficient matrix offers effective dimension reduction, which greatly facilitates parameter estimation and model interpretation. However, commonly used reduced-rank methods are sensitive to data corruption, as the low-rank dependence structure between response variables and predictors is easily distorted by outliers. We propose a robust reduced-rank regression approach for joint modelling and outlier detection. The problem is formulated as a regularized multivariate regression with a sparse mean-shift parameterization, which generalizes and unifies some popular robust multivariate methods. An efficient thresholding-based iterative procedure is developed for optimization. We show that the algorithm is guaranteed to converge and that the coordinatewise minimum point produced is statistically accurate under regularity conditions. Our theoretical investigations focus on non-asymptotic robust analysis, demonstrating that joint rank reduction and outlier detection leads to improved prediction accuracy. In particular, we show that redescending -functions can essentially attain the minimax optimal error rate, and in some less challenging problems convex regularization guarantees the same low error rate. The performance of the proposed method is examined through simulation studies and real-data examples.
Keywords
All Related Versions
Funding Information
- National Institutes of Health
This publication has 27 references indexed in Scilit:
- Reduced rank ridge regression and its kernel extensionsStatistical Analysis and Data Mining: An ASA Data Science Journal, 2011
- Robust principal component analysis?Journal of the ACM, 2011
- Estimation of high-dimensional low-rank matricesThe Annals of Statistics, 2011
- Optimal selection of reduced rank estimators of high-dimensional matricesThe Annals of Statistics, 2011
- Discovering genetic associations with high-dimensional neuroimaging phenotypes: A sparse reduced-rank regression approachNeuroImage, 2010
- Nearly unbiased variable selection under minimax concave penaltyThe Annals of Statistics, 2010
- Dimension Reduction and Coefficient Estimation in Multivariate Linear RegressionJournal of the Royal Statistical Society Series B: Statistical Methodology, 2007
- On the uniqueness of S-functionals and M-functionals under nonelliptical distributionsThe Annals of Statistics, 2000
- A Fast Algorithm for the Minimum Covariance Determinant EstimatorTechnometrics, 1999
- Estimating Linear Restrictions on Regression Coefficients for Multivariate Normal DistributionsThe Annals of Mathematical Statistics, 1951