Incorporating selective victim cache into GPGPU for high‐performance computing
- 3 March 2017
- journal article
- research article
- Published by Wiley in Concurrency and Computation: Practice and Experience
- Vol. 29 (24)
Abstract
Summary: Contemporary general‐purpose graphic processing units (GPGPUs) successfully parallelize an application into thousands of concurrent threads with remarkably improved performance. Such massive threads will compete for the small‐sized first‐level data (L1D) cache, leading to an exaggerated cache‐thrashing problem, which may degrade the overall performance significantly. In this paper, we propose a selective victim cache design to enable better data locality and higher performance. Instead of a small fully associative structure, we first redesign the victim cache as a set associative structure that is equivalent to the original L1D cache to suit the GPGPU applications with massive concurrent threads. To keep the mostly used data in L1D for better operand service, we apply a simple prediction scheme to avoid costly block interchanges and evictions. To further save the area for data storage, we propose to leverage the unallocated registers and shared memory entries to hold the victim cache data. The experiments demonstrate that our proposed approach can increase the on‐chip data cache hit rate considerably and deliver a better performance with negligible changes to the baseline GPGPU architecture. For example, our selective victim cache design can improve the performance by 41.3% on average, achieving 54.7% increase in data cache hit rate and 21.8% reduction in block interchanges and evictions.Keywords
Funding Information
- National Natural Science Foundation of China (61202026, 61332001, 61402285)
This publication has 27 references indexed in Scilit:
- Efficient utilization of GPGPU cache hierarchy2015
- Cache-Conscious Wavefront Scheduling2012
- Characterizing and improving the use of demand-fetched caches in GPUs2012
- Optimizing MapReduce for GPUs with effective shared memory usage2012
- CudaDMA2011
- Understanding GPU Programming for Statistical Computation: Studies in Massively Parallel Massive MixturesJournal of Computational and Graphical Statistics, 2010
- Automatic C-to-CUDA Code Generation for Affine Programs2010
- Optimization principles and application performance evaluation of a multithreaded GPU using CUDA2008
- Selective victim caching: a method to improve the performance of direct-mapped cachesIEEE Transactions on Computers, 1997
- Improving direct-mapped cache performance by the addition of a small fully-associative cache and prefetch buffersACM SIGARCH Computer Architecture News, 1990