Improving GPGPU concurrency with elastic kernels
Authors:
Open Access
- 16 March 2013
- journal article
- research article
- Published by Association for Computing Machinery (ACM) in ACM SIGPLAN Notices
- Vol. 48 (4) , 407-418
Abstract
Each new generation of GPUs vastly increases the resources available to GPGPU programs. GPU programming models (like CUDA) were designed to scale to use these resources. However, we find that CUDA programs actually do not scale to utilize all available resources, with over 30% of resources going unused on average for programs of the Parboil2 suite that we used in our work. Current GPUs therefore allow concurrent execution of kernels to improve utilization. In this work, we study concurrent execution of GPU kernels using multiprogram workloads on current NVIDIA Fermi GPUs. On two-program workloads from the Parboil2 benchmark suite we find concurrent execution is often no better than serialized execution. We identify that the lack of control over resource allocation to kernels is a major serialization bottleneck. We propose transformations that convert CUDA kernels into elastic kernels which permit fine-grained control over their resource usage. We then propose several elastic-kernel aware concurrency policies that offer significantly better performance and concurrency compared to the current CUDA policy. We evaluate our proposals on real hardware using multiprogrammed workloads constructed from benchmarks in the Parboil 2 suite. On average, our proposals increase system throughput (STP) by 1.21x and improve the average normalized turnaround time (ANTT) by 3.73x for two-program workloads when compared to the current CUDA concurrency implementation.Keywords
This publication has 10 references indexed in Scilit:
- The case for GPGPU spatial multitasking2012
- Supporting GPU sharing in cloud environments with a transparent runtime consolidation framework2011
- Kernel Fusion: An Effective Method for Better Power Efficiency on Multithreaded GPU2010
- Efficient compilation of fine-grained SPMD-threaded programs for multicore CPUs2010
- Rodinia: A benchmark suite for heterogeneous computing2009
- Chunking parallel loops in the presence of synchronization2009
- Analyzing CUDA workloads using a detailed GPU simulator2009
- System-Level Performance Metrics for Multiprogram WorkloadsIEEE Micro, 2008
- NVIDIA Tesla: A Unified Graphics and Computing ArchitectureIEEE Micro, 2008
- MCUDA: An Efficient Implementation of CUDA Kernels for Multi-core CPUs2008