Warped-Slicer: Efficient Intra-SM Slicing through Dynamic Resource Partitioning for GPU Multiprogramming
Authors:
- Qiumin XuQXQiumin Xu
- Hyeran JeonHJHyeran Jeon
- Keunsoo KimKKKeunsoo Kim
- Won Woo RoWRWon Woo Ro
- Murali AnnavaramMAMurali Annavaram
- 25 August 2016
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- No. 10636897,p. 230-242
Abstract
As technology scales, GPUs are forecasted to incorporate an ever-increasing amount of computing resources to support thread-level parallelism. But even with the best effort, exposing massive thread-level parallelism from a single GPU kernel, particularly from general purpose applications, is going to be a difficult challenge. In some cases, even if there is sufficient thread-level parallelism in a kernel, there may not be enough available memory bandwidth to support such massive concurrent thread execution. Hence, GPU resources may be underutilized as more general purpose applications are ported to execute on GPUs. In this paper, we explore multiprogramming GPUs as a way to resolve the resource underutilization issue. There is a growing hardware support for multiprogramming on GPUs. Hyper-Q has been introduced in the Kepler architecture which enables multiple kernels to be invoked via tens of hardware queue streams. Spatial multitasking has been proposed to partition GPU resources across multiple kernels. But the partitioning is done at the coarse granularity of streaming multiprocessors (SMs) where each kernel is assigned to a subset of SMs. In this paper, we advocate for partitioning a single SM across multiple kernels, which we term as intra-SM slicing. We explore various intra-SM slicing strategies that slice resources within each SM to concurrently run multiple kernels on the SM. Our results show that there is not one intra-SM slicing strategy that derives the best performance for all application pairs. We propose Warped-Slicer, a dynamic intra-SM slicing strategy that uses an analytical method for calculating the SM resource partitioning across different kernels that maximizes performance. The model relies on a set of short online profile runs to determine how each kernel's performance varies as more thread blocks from each kernel are assigned to an SM. The model takes into account the interference effect of shared resource usage across multiple kernels. The model is also computationally efficient and can determine the resource partitioning quickly to enable dynamic decision making as new kernels enter the system. We demonstrate that the proposed Warped-Slicer approach improves performance by 23% over the baseline multiprogramming approach with minimal hardware overhead.Keywords
This publication has 41 references indexed in Scilit:
- Virtual Thread: Maximizing Thread-Level Parallelism beyond GPU Scheduling Limit2016
- Cache QoS: From concept to reality in the Intel® Xeon® processor E5-2600 v3 product family2016
- Futility Scaling: High-Associativity Cache Partitioning2014
- Equalizer: Dynamic Tuning of GPU Resources for Efficient Execution2014
- Many-Core vs. Many-Thread Machines: Stay Away From the ValleyIEEE Computer Architecture Letters, 2009
- Parallelism-Aware Batch SchedulingACM SIGARCH Computer Architecture News, 2008
- Utility-Based Cache Partitioning: A Low-Overhead, High-Performance, Runtime Mechanism to Partition Shared Caches40th Annual IEEE/ACM International Symposium on Microarchitecture (MICRO 2007), 2006
- Learning-Based SMT Processor Resource Distribution via Hill-Climbing2006
- Predictable performance in SMT processors: synergy between the OS and SMTsIEEE Transactions on Computers, 2006
- Symbiotic jobscheduling for a simultaneous multithreaded processor2000