WASP: Exploiting GPU Pipeline Parallelism with Hardware-Accelerated Automatic Warp Specialization
Authors:
- 2 March 2024
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 1-16
Abstract
Graphics processing units (GPUs) are an important class of parallel processors that offer high compute throughput and memory bandwidth. GPUs are used in a variety of important computing domains, such as machine learning, high performance computing, sparse linear algebra, autonomous vehicles, and robotics. However, some applications from these domains can underperform due to sensitivity to memory latency and bandwidth. Some of this sensitivity can be reduced by better overlapping memory access with compute. Current GPUs often leverage pipeline parallelism in the form of warp specialization to enable better overlap. However, current warp specialization support on GPUs is limited in three ways. First, warp specialization is a complex and manual program transformation that is out of reach for many applications and developers. Second, it is limited to coarse-grained transfers between global memory and the shared memory scratchpad (SMEM); fine-grained memory access patterns are not well supported. Finally, the GPU hardware is unaware of the pipeline parallelism expressed by the programmer, and is unable to take advantage of this information to make better decisions at runtime. In this paper we introduce WASP, hardware and compiler support for warp specialization that addresses these limitations. WASP enables fine-grained streaming and gather memory access patterns through the use of warp-level register file queues and hardware-accelerated address generation. Explicit warp to pipeline stage naming enables the GPU to be aware of pipeline parallelism, which WASP capitalizes on by designing pipeline-aware warp mapping, register allocation, and scheduling. Finally, we design and implement a compiler that can automatically generate warp specialized kernels, reducing programmer burden. Overall, we find that runtime performance can be improved on a variety of important applications by an average of 47% over a modern GPU baseline.Keywords
This publication has 31 references indexed in Scilit:
- CuRobo: Parallelized Collision-Free Robot Motion Generation2023
- Phloem: Automatic Acceleration of Irregular Applications with Fine-Grain Pipeline Parallelism2023
- Automatic Horizontal Fusion for GPU Kernels2022
- Need for Speed: Experiences Building a Trustworthy System-Level GPU Simulator2021
- Efficient Non-fused Winograd on GPUs2020
- CTA-Aware Prefetching and Scheduling for GPU2018
- Mascar: Speeding up GPU warps by reducing memory pitstops2015
- A quantitative study of irregular programs on GPUs2012
- Achieving Out-of-Order Performance with Almost In-Order Complexity2008
- StreamIt: A Language for Streaming Applications2002