SLOE: A Faster Method for Statistical Inference in High-Dimensional Logistic Regression
Authors:
- Steve YadlowskySYSteve Yadlowsky
- Taedong YunTYTaedong Yun
- Cory McLeanCMCory McLean
- Alexander D'AmourADAlexander D'Amour
Preprint
Abstract
Logistic regression remains one of the most widely used tools in applied statistics, machine learning and data science. However, in moderately high-dimensional problems, where the number of features $d$ is a non-negligible fraction of the sample size $n$, the logistic regression maximum likelihood estimator (MLE), and statistical procedures based the large-sample approximation of its distribution, behave poorly. Recently, Sur and Cand\`es (2019) showed that these issues can be corrected by applying a new approximation of the MLE's sampling distribution in this high-dimensional regime. Unfortunately, these corrections are difficult to implement in practice, because they require an estimate of the \emph{signal strength}, which is a function of the underlying parameters $\beta$ of the logistic regression. To address this issue, we propose SLOE, a fast and straightforward approach to estimate the signal strength in logistic regression. The key insight of SLOE is that the Sur and Cand\`es (2019) correction can be reparameterized in terms of the \emph{corrupted signal strength}, which is only a function of the estimated parameters $\widehat \beta$. We propose an estimator for this quantity, prove that it is consistent in the relevant high-dimensional regime, and show that dimensionality correction using SLOE is accurate in finite samples. Compared to the existing ProbeFrontier heuristic, SLOE is conceptually simpler and orders of magnitude faster, making it suitable for routine use. We demonstrate the importance of routine dimensionality correction in the Heart Disease dataset from the UCI repository, and a genomics application using data from the UK Biobank. We provide an open source package for this method, available at \url{https://github.com/google-research/sloe-logistic}.
Keywords
All Related Versions
This publication has 11 references indexed in Scilit:
- Large uncertainty in individual polygenic risk score estimation impacts PRS-based risk stratificationNature Genetics, 2021
- Large-scale machine-learning-based phenotyping significantly improves genomic discovery for optic nerve head morphologyThe American Journal of Human Genetics, 2021
- Second opinion needed: communicating uncertainty in medical machine learningnpj Digital Medicine, 2021
- Array programming with NumPyNature, 2020
- SciPy 1.0: fundamental algorithms for scientific computing in PythonNature Methods, 2020
- Gender in cardiovascular medicine: chest pain and coronary artery diseaseEuropean Heart Journal, 2019
- On the impact of predictor geometry on the performance on high-dimensional ridge-regularized generalized robust regression estimatorsProbability Theory and Related Fields, 2017
- Controlling the False Discovery Rate: A Practical and Powerful Approach to Multiple TestingJournal of the Royal Statistical Society Series B: Statistical Methodology, 1995
- International application of a new probability algorithm for the diagnosis of coronary artery diseaseThe American Journal of Cardiology, 1989
- Adjustment of an Inverse Matrix Corresponding to a Change in One Element of a Given MatrixThe Annals of Mathematical Statistics, 1950