Dissecting the CUDA scheduling hierarchy: a Performance and Predictability Perspective
- 1 April 2020
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 213-225
Abstract
Over the last few years, the ever-increasing use of Graphic Processing Units (GPUs) in safety-related domains has opened up many research problems in the real-time community. The closed and proprietary nature of the scheduling mechanisms deployed in NVIDIA GPUs, for instance, represents a major obstacle in deriving a proper schedulability analysis for latency-sensitive applications. Existing literature addresses these issues by either (i) providing simplified models for heterogeneous CPUGPU systems and their associated scheduling policies, or (ii) providing insights about these arbitration mechanisms obtained through reverse engineering. In this paper, we take one step further by correcting and consolidating previously published assumptions about the hierarchical scheduling policies of NVIDIA GPUs and their proprietary CUDA application programming interface. We also discuss how such mechanisms evolved with recently released GPU micro-architectures, and how such changes influence the scheduling models to be exploited by real-time system engineers.Keywords
This publication has 19 references indexed in Scilit:
- Maximizing the GPU resource usage by reordering concurrent kernels submissionConcurrency and Computation: Practice and Experience, 2018
- SiGAMMA2017
- Memory interference characterization between CPU cores and integrated GPUs in mixed-criticality platforms2017
- Rethinking the Inception Architecture for Computer Vision2016
- Performance modeling in CUDA streams — A means for high-throughput data processing2014
- Preemptive thread block scheduling with online structural runtime prediction for concurrent GPGPU kernels2014
- Kernelet: High-Throughput GPU Kernel Executions with Dynamic Slicing and SchedulingIEEE Transactions on Parallel and Distributed Systems, 2013
- Increasing GPU throughput using kernel interleaved thread block scheduling2013
- Improving GPGPU concurrency with elastic kernelsACM SIGPLAN Notices, 2013
- On Improving the Performance of Multi-threaded CUDA Applications with Concurrent Kernel Execution by Kernel Reordering2012