Ripple: Profile-Guided Instruction Cache Replacement for Data Center Applications
Authors:
- 1 June 2021
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 734-747
Abstract
Modern data center applications exhibit deep software stacks, resulting in large instruction footprints that frequently cause instruction cache misses degrading performance, cost, and energy efficiency. Although numerous mechanisms have been proposed to mitigate instruction cache misses, they still fall short of ideal cache behavior, and furthermore, introduce significant hardware overheads. We first investigate why existing I-cache miss mitigation mechanisms achieve sub-optimal performance for data center applications. We find that widely-studied instruction prefetchers fall short due to wasteful prefetch-induced cache line evictions that are not handled by existing replacement policies. Existing replacement policies are unable to mitigate wasteful evictions since they lack complete knowledge of a data center application’s complex program behavior.To make existing replacement policies aware of these eviction-inducing program behaviors, we propose Ripple, a novel software-only technique that profiles programs and uses program context to inform the underlying replacement policy about efficient replacement decisions. Ripple carefully identifies program con-texts that lead to I-cache misses and sparingly injects "cache line eviction" instructions in suitable program locations at link time. We evaluate Ripple using nine popular data center applications and demonstrate that Ripple enables any replacement policy to achieve speedup that is closer to that of an ideal I-cache. Specifically, Ripple achieves an average performance improvement of 1.6% (up to 2.13%) over prior work due to a mean 19% (up to 28.6%) I-cache miss reduction.Keywords
Funding Information
- Intel Corporation
This publication has 72 references indexed in Scilit:
- A Primer on Hardware PrefetchingSynthesis Lectures on Computer Architecture, 2014
- High performance cache replacement using re-reference interval prediction (RRIP)ACM SIGARCH Computer Architecture News, 2010
- Adaptive insertion policies for high performance cachingACM SIGARCH Computer Architecture News, 2007
- IATAC: a smart predictor to turn-off L2 cache linesACM Transactions on Architecture and Code Optimization, 2005
- Timekeeping in the memory systemACM SIGARCH Computer Architecture News, 2002
- EELRUACM SIGMETRICS Performance Evaluation Review, 1999
- The LRU-K page replacement algorithm for database disk bufferingACM SIGMOD Record, 1993
- Reducing memory latency via non-blocking and prefetching cachesACM SIGPLAN Notices, 1992
- Evaluating associativity in CPU cachesIEEE Transactions on Computers, 1989
- Cache MemoriesACM Computing Surveys, 1982