Memory access patterns
Authors:
Open Access
- 15 November 2015
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 1-12
Abstract
With the increased popularity of multi-GPU nodes in modern HPC clusters, it is imperative to develop matching programming paradigms for their efficient utilization. In order to take advantage of the local GPUs and the low-latency high-throughput interconnects that link them, programmers need to meticulously adapt parallel applications with respect to load balancing, boundary conditions and device synchronization. This paper presents MAPS-Multi, an automatic multi-GPU partitioning framework that distributes the workload based on the underlying memory access patterns. The framework consists of host- and device-level APIs that allow programs to efficiently run on a variety of GPU and multi-GPU architectures. The framework implements several layers of code optimization, device abstraction, and automatic inference of inter-GPU memory exchanges. The paper demonstrates that the performance of MAPS-Multi achieves near-linear scaling on fundamental computational operations, as well as real-world applications in deep learning and multivariate analysis.Keywords
Funding Information
- Ministry of Science and Technology, Israel
- Deutsche Forschungsgemeinschaft
This publication has 22 references indexed in Scilit:
- MAPSACM Transactions on Architecture and Code Optimization, 2014
- High performance MRI simulations of motion on multi-GPU systemsJournal of Cardiovascular Magnetic Resonance, 2014
- Automatic data allocation and buffer management for multi-GPU machinesACM Transactions on Architecture and Code Optimization, 2013
- Shallow Water Simulations on Multiple GPUs2012
- A Multi-GPU Programming Library for Real-Time Applications2012
- Parallel Programmability and the Chapel LanguageThe International Journal of High Performance Computing Applications, 2007
- Metagenes and molecular pattern discovery using matrix factorizationProceedings of the National Academy of Sciences of the United States of America, 2004
- Gradient-based learning applied to document recognitionProceedings of the IEEE, 1998
- High performance Fortran language specificationACM SIGPLAN Fortran Forum, 1993
- Mathematical GamesScientific American, 1970