Overcoming Memory Capacity Wall of GPUs With Heterogeneous Memory Stack
- 8 August 2022
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Computer Architecture Letters
- Vol. 21 (2) , 61-64
Abstract
We propose to overcome the memory capacity limitation of GPUs with a Heterogeneous Memory Stack (HMS) that integrates Storage Class Memory (SCM) and DRAM in a 3D memory stack. By effectively utilizing the DRAM as a cache, the HMS addresses the latency, bandwidth, and energy challenges of the SCM. For workloads that mandate memory oversubscription, the HMS can capture a larger fraction of the memory footprint compared to an HBM, providing a significant performance uplift. To enable a performant DRAM cache architecture, we propose a DRAM cache bypass mechanism that avoids DRAM cache thrashing by 100,000 s of GPU threads. Furthermore, we propose a Tag Cache which uses part of L2 cache to hold tags of DRAM cachelines and reduce DRAM cache probe traffic, while enabling adjustment of L2 cache capacity used for Tag Cache. In addition, our DRAM cache organization enables fetching all DRAM cacheline tags in a row with a single column access, avoiding excessive DRAM cache probe traffic from Tag Cache misses. Our results show that the HMS outperforms the baseline HBM system under memory oversubscription by up to 9.8× (2.8× on average).Keywords
Funding Information
- IITP (2021-0-00871)
This publication has 19 references indexed in Scilit:
- Oversubscribing GPU Unified Virtual Memory2022
- FPnew: An Open-Source Multiformat Floating-Point Unit Architecture for Energy-Proportional Transprecision ComputingIEEE Transactions on Very Large Scale Integration (VLSI) Systems, 2020
- Accel-Sim: An Extensible Simulation Framework for Validated GPU Modeling2020
- A Framework for Memory Oversubscription Management in Graphics Processing Units2019
- Design guidelines for high-performance SCM hierarchies2018
- Combining HW/SW Mechanisms to Improve NUMA Performance of Multi-GPU Systems2018
- CACTI 7ACM Transactions on Architecture and Code Optimization, 2017
- GraphBIG2015
- BEAR2015
- Ramulator: A Fast and Extensible DRAM SimulatorIEEE Computer Architecture Letters, 2015