Automatic Parallelization of Kernels in Shared-Memory Multi-GPU Nodes
Authors:
Open Access
- 8 June 2015
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 3-13
Abstract
In this paper we present AMGE, a programming framework and runtime system that transparently decomposes GPU kernels and executes them on multiple GPUs in parallel. AMGE exploits the remote memory access capability in modern GPUs to ensure that data can be accessed regardless of its physical location, allowing our runtime to safely decompose and distribute arrays across GPU memories. It optionally performs a compiler analysis that detects array access patterns in GPU kernels. Using this information, the runtime can perform more efficient computation and data distribution configurations than previous works. The GPU execution model allows AMGE to hide the cost of remote accesses if they are kept below 5%. We demonstrate that a thread block scheduling policy that distributes remote accesses through the whole kernel execution further reduces their overhead. Results show 1.98× and 3.89× execution speedups for 2 and 4 GPUs for a wide range of dense computations compared to the original versions on a single GPU.Keywords
Funding Information
- Spanish Ministry of Science and Technology (TIN2012-34557)
- Spanish Government (SEV-2011-0067)
This publication has 24 references indexed in Scilit:
- Comparison based sorting for systems with multiple GPUs2013
- GA-GPU2012
- Habanero-Java2011
- Feedback-directed page placement for ccNUMA via hardware-generated memory tracesJournal of Parallel and Distributed Computing, 2010
- GPU-accelerated molecular modeling coming of ageJournal of Molecular Graphics and Modelling, 2010
- Implementation of a Lattice Boltzmann kernel using the Compute Unified Device Architecture developed by nVIDIAComputing and Visualization in Science, 2008
- Data and computation transformations for multiprocessors1995
- The High Performance Fortran HandbookComputers in Physics, 1994
- Demonstration of automatic data partitioning techniques for parallelizing compilers on multicomputersIEEE Transactions on Parallel and Distributed Systems, 1992
- Compiling programs for distributed-memory multiprocessorsThe Journal of Supercomputing, 1988