Comparative Study of CUDA GPU Implementations in Python With the Fast Iterative Shrinkage-Thresholding Algorithm for LASSO
Open Access
- 18 May 2022
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 10, 53324-53343
Abstract
A general-purpose GPU (GPGPU) is employed in a variety of domains, including accelerating the spread of deep natural network models; however, further research into its effective implementation is needed. When using the compute unified device architecture (CUDA), which has recently gained popularity, the situation is analogous to use the GPUs and its memory space. This is due to the lack of a gold standard for selecting the most efficient approach for CUDA GPU parallel computation. Contrarily, as solving the least absolute shrinkage and selection operator (LASSO) regression fully consists of the basic linear algebra operations, the computation using GPGPU is more effective than other models. Additionally, its optimization problem often requires fast and efficient calculations. The purpose of this study is to provide brief introductions to the implementation approaches and numerically compare the computational efficiency of GPU parallel computation with that of the fast iterative shrinkage-thresholding algorithm for LASSO. This study contributes to providing gold standards for the CUDA GPU parallel computation, considering both computational efficiency and ease of implementation. Based on our comparison results, we recommend implementing the CUDA GPU parallel computation using Python, with either a dynamic-link library or PyTorch for the iterative algorithms.Keywords
Funding Information
- Inha University Research Grant
- National Research Foundation of Korea (NRF-2020R1F1A1A01048127)
This publication has 19 references indexed in Scilit:
- Numba2015
- CUDA ClustalW: An efficient parallel algorithm for progressive multiple sequence alignment on Multi-GPUsComputational Biology and Chemistry, 2015
- Adaptive block size for dense QR factorization in hybrid CPU–GPU systems via statistical modelingParallel Computing, 2014
- PyCUDA and PyOpenCL: A scripting-based approach to GPU run-time code generationParallel Computing, 2012
- The solution path of the generalized lassoThe Annals of Statistics, 2011
- OpenCL: A Parallel Programming Standard for Heterogeneous Computing SystemsComputing in Science & Engineering, 2010
- A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse ProblemsSIAM Journal on Imaging Sciences, 2009
- Coordinate descent algorithms for lasso penalized regressionThe Annals of Applied Statistics, 2008
- An iterative thresholding algorithm for linear inverse problems with a sparsity constraintCommunications on Pure and Applied Mathematics, 2004
- Least angle regressionThe Annals of Statistics, 2004