CUDA-NP
Authors:
Open Access
- 6 February 2014
- journal article
- research article
- Published by Association for Computing Machinery (ACM) in ACM SIGPLAN Notices
- Vol. 49 (8) , 93-106
Abstract
Parallel programs consist of series of code sections with different thread-level parallelism (TLP). As a result, it is rather common that a thread in a parallel program, such as a GPU kernel in CUDA programs, still contains both se-quential code and parallel loops. In order to leverage such parallel loops, the latest Nvidia Kepler architecture intro-duces dynamic parallelism, which allows a GPU thread to start another GPU kernel, thereby reducing the overhead of launching kernels from a CPU. However, with dynamic parallelism, a parent thread can only communicate with its child threads through global memory and the overhead of launching GPU kernels is non-trivial even within GPUs. In this paper, we first study a set of GPGPU benchmarks that contain parallel loops, and highlight that these bench-marks do not have a very high loop count or high degrees of TLP. Consequently, the benefits of leveraging such par-allel loops using dynamic parallelism are too limited to offset its overhead. We then present our proposed solution to exploit nested parallelism in CUDA, referred to as CUDA-NP. With CUDA-NP, we initially enable a high number of threads when a GPU program starts, and use control flow to activate different numbers of threads for different code sections. We implemented our proposed CUDA-NP framework using a directive-based compiler approach. For a GPU kernel, an application developer only needs to add OpenMP-like pragmas for parallelizable code sections. Then, our CUDA-NP compiler automatically gen-erates the optimized GPU kernels. It supports both the reduction and the scan primitives, explores different ways to distribute parallel loop iterations into threads, and effi-ciently manages on-chip resource. Our experiments show that for a set of GPGPU benchmarks, which have already been optimized and contain nested parallelism, our pro-posed CUDA-NP framework further improves the perfor-mance by up to 6.69 times and 2.18 times on average.Keywords
This publication has 35 references indexed in Scilit:
- A performance analysis framework for identifying potential benefits in GPGPU applications2012
- Improving GPU performance via large warps and two-level warp scheduling2011
- Dynamic Detection of Uniform and Affine Vectors in GPGPU Computations2010
- OpenMP to GPGPU2009
- CUDA-Lite: Reducing GPU Programming Complexity2008
- A Microbenchmark Study of OpenMP Overheads under Nested ParallelismLecture Notes in Computer Science, 2008
- A compiler for exploiting nested parallelism in OpenMP programsParallel Computing, 2005
- Cetus – An Extensible Compiler Infrastructure for Source-to-Source Transformation2004
- Performance Evaluation of OpenMP Applications with Nested ParallelismLecture Notes in Computer Science, 2000
- OpenMP: an industry standard API for shared-memory programmingIEEE Computational Science and Engineering, 1998