Stream Floating: Enabling Proactive and Decentralized Cache Optimizations
Authors:
- 1 February 2021
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 640-653
Abstract
As multicore systems continue to grow in scale and on-chip memory capacity, the on-chip network bandwidth and latency become problematic bottlenecks. Because of this, overheads in data transfer, the coherence protocol and replacement policies become increasingly important. Unfortunately, even in well-structured programs, many natural optimizations are difficult to implement because of the reactive and centralized nature of traditional cache hierarchies, where all requests are initiated by the core for short, cache line granularity accesses. For example, long-lasting access patterns could be streamed from shared caches without requests from the core. Indirect memory access can be performed by chaining requests made from within the cache, rather than constantly returning to the core. Our primary insight is that if programs can embed information about long-term memory stream behavior in their ISAs, then these streams can be floated to the appropriate level of the memory hierarchy. This decentralized approach to address generation and cache requests can lead to better cache policies and lower request and data traffic by proactively sending data before the cores even request it. To evaluate the opportunities of stream floating, we enhance a tiled multicore cache hierarchy with stream engines to process stream requests in last-level cache banks. We develop several novel optimizations that are facilitated by stream exposure in the ISA, and subsequent exposure to caches. We evaluate using a cycle-level execution-driven gem5-based simulator, using 10 data-processing workloads from Rodinia and 2 streaming kernels written in OpenMP. We find that stream floating enables 52% and 39% speedup over an inorder and OOO core with state of art prefetcher design respectively, with 64% and 49% energy efficiency advantage.Keywords
This publication has 54 references indexed in Scilit:
- PIM-enabled instructions2015
- The Direct-to-Data (D2D) cacheACM SIGARCH Computer Architecture News, 2014
- DeFTACM Transactions on Architecture and Code Optimization, 2011
- The gem5 simulatorACM SIGARCH Computer Architecture News, 2011
- High performance cache replacement using re-reference interval prediction (RRIP)ACM SIGARCH Computer Architecture News, 2010
- Adaptive insertion policies for high performance cachingACM SIGARCH Computer Architecture News, 2007
- RegionScoutACM SIGARCH Computer Architecture News, 2005
- Timekeeping in the memory systemACM SIGARCH Computer Architecture News, 2002
- Evaluation of a Competitive-Update Cache Coherence Protocol with Migratory Data DetectionJournal of Parallel and Distributed Computing, 1996
- Adaptive cache coherency for detecting migratory shared dataACM SIGARCH Computer Architecture News, 1993