RPG 2 : Robust Profile-Guided Runtime Prefetch Generation
Open Access
- 27 April 2024
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 999-1013
Abstract
Data cache prefetching is a well-established optimization to overcome the limits of the cache hierarchy and keep the processor pipeline fed with data. In principle, accurate, well-timed prefetches can sidestep the majority of cache misses and dramatically improve performance. In practice, however, it is challenging to identify which data to prefetch and when to do so. In particular, data can be easily requested too early, causing eviction of useful data from the cache, or requested too late, failing to avoid cache misses. Competition for limited off-chip memory bandwidth must also be balanced between prefetches and a program's regular "demand" accesses. Due to these challenges, prefetching can both help and hurt performance, and the outcome can depend on program structure, decisions about what to prefetch and when to do it, and, as we demonstrate in a series of experiments, program input, processor microarchitecture, and their interaction as well. To try to meet these challenges, we have designed the RPG2 system for online prefetch injection and tuning. RPG2 is a pure-software system that operates on running C/C++ programs, profiling them, injecting prefetch instructions, and then tuning those prefetches to maximize performance. Across dozens of inputs, we find that RPG2 can provide speedups of up to 2.15×, comparable to the best profile-guided prefetching compilers, but can also respond when prefetching ends up being harmful and roll back to the original code - something that static compilers cannot. RPG2 improves prefetching robustness by preserving its performance benefits, while avoiding slowdowns.Keywords
Funding Information
- NSF/Intel joint grant (#2011168)
- NSF (National Science Foundation) (#1942754, #2346057, #1841545)
- the PRISM Research Center
- a JUMP Center cosponsored by SRC and DARPA
- Intel Labs, the Intel TSA project
This publication has 55 references indexed in Scilit:
- Prodigy: Improving the Memory Latency of Data-Indirect Irregular Workloads Using Hardware-Software Co-Design2021
- Analysis and Optimization of the Memory Hierarchy for Graph Processing Workloads2019
- BOLT: A Practical Binary Optimizer for Data Centers and Beyond2019
- CRONO: A Benchmark Suite for Multithreaded Graph Algorithms Executing on Futuristic Multicores2015
- A Case for Resource Efficient Prefetching in Multicores2014
- Runahead Threads to improve SMT performance2008 IEEE 14th International Symposium on High Performance Computer Architecture, 2008
- Accelerating and Adapting Precomputation Threads for Effcient Prefetching2007
- Runahead execution: an alternative to very large instruction windows for out-of-order processors2003
- Effectiveness of hardware-based stride and sequential prefetching in shared-memory multiprocessors2002
- Sequential Program Prefetching in Memory HierarchiesComputer, 1978