Open Access
- 4 September 2018
- journal article
- research article
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Architecture and Code Optimization
- Vol. 15 (3) , 1-23
Abstract
To exploit parallelism and scalability of multiple GPUs in a system, it is critical to place compute and data together. However, two key techniques that have been used to hide memory latency and improve thread-level parallelism (TLP), memory interleaving, and thread block scheduling, in traditional GPU systems are at odds with efficient use of multiple GPUs. Distributing data across multiple GPUs to improve overall memory bandwidth utilization incurs high remote traffic when the data and compute are misaligned. Nondeterministic thread block scheduling to improve compute resource utilization impedes co-placement of compute and data. Our goal in this work is to enable co-placement of compute and data in the presence of fine-grained interleaved memory with a low-cost approach. To this end, we propose a mechanism that identifies exclusively accessed data and place the data along with the thread block that accesses it in the same GPU. The key ideas are (1) the amount of data exclusively used by a thread block can be estimated, and that exclusive data (of any size) can be localized to one GPU with coarse-grained interleaved pages; (2) using the affinity-based thread block scheduling policy, we can co-place compute and data together; and (3) by using dual address mode with lightweight changes to virtual to physical page mappings, we can selectively choose different interleaved memory pages for each data structure. Our evaluations across a wide range of workloads show that the proposed mechanism improves performance by 31% and reduces 38% remote traffic over a baseline system.Keywords
This publication has 43 references indexed in Scilit:
- Data reorganization in memory using 3D-stacked DRAM2015
- A scalable processing-in-memory accelerator for parallel graph processing2015
- Active Memory Cube: A processing-in-memory architecture for exascale systemsIBM Journal of Research and Development, 2015
- Automatic data allocation and buffer management for multi-GPU machinesACM Transactions on Architecture and Code Optimization, 2013
- The structural simulation toolkitACM SIGMETRICS Performance Evaluation Review, 2011
- The Characterization of Data Intensive Memory Workloads on Distributed PIM Systems⋆2001
- Computational RAM: implementing processors in memoryIEEE Design & Test of Computers, 1999
- A case for intelligent RAMIEEE Micro, 1997
- Processing in memory: the Terasys massively parallel PIM arrayComputer, 1995
- High performance Fortran language specificationACM SIGPLAN Fortran Forum, 1993