High-Performance Statistical Computing in the Computing Environments of the 2020s
Preprint
Abstract
Technological advances in the past decade, hardware and software alike, have made access to high-performance computing (HPC) easier than ever. We review these advances from a statistical computing perspective. Cloud computing makes access to supercomputers affordable. Deep learning software libraries make programming statistical algorithms easy and enable users to write code once and run it anywhere -- from a laptop to a workstation with multiple graphics processing units (GPUs) or a supercomputer in a cloud. Highlighting how these developments benefit statisticians, we review recent optimization algorithms that are useful for high-dimensional models and can harness the power of HPC. Code snippets are provided to demonstrate the ease of programming. We also provide an easy-to-use distributed matrix data structure suitable for HPC. Employing this data structure, we illustrate various statistical applications including large-scale positron emission tomography and $\ell_1$-regularized Cox regression. Our examples easily scale up to an 8-GPU workstation and a 720-CPU-core cluster in a cloud. As a case in point, we analyze the onset of type-2 diabetes from the UK Biobank with 200,000 subjects and about 500,000 single nucleotide polymorphisms using the HPC $\ell_1$-regularized Cox regression. Fitting this half-million-variate model takes less than 45 minutes and reconfirms known associations. To our knowledge, this is the first demonstration of the feasibility of penalized regression of survival outcomes at this scale.Keywords
All Related Versions
This publication has 82 references indexed in Scilit:
- Convergence of Stochastic Proximal Gradient AlgorithmApplied Mathematics & Optimization, 2019
- Monotone operator theory in convex optimizationMathematical Programming, 2018
- Parallel coordinate descent methods for big data optimizationMathematical Programming, 2015
- A Primal–Dual Splitting Method for Convex Optimization Involving Lipschitzian, Proximable and Linear Composite TermsJournal of Optimization Theory and Applications, 2012
- A splitting algorithm for dual monotone inclusions involving cocoercive operatorsAdvances in Computational Mathematics, 2011
- A First-Order Primal-Dual Algorithm for Convex Problems with Applications to ImagingJournal of Mathematical Imaging and Vision, 2010
- A coordinate gradient descent method for nonsmooth separable minimizationMathematical Programming, 2007
- SLURM: Simple Linux Utility for Resource Management2003
- SUMMA: scalable universal matrix multiplication algorithmConcurrency: Practice and Experience, 1997
- A nonsmooth version of Newton's methodMathematical Programming, 1993