FLASH: Deadline-Aware Flexible LLC Arbitration and Scheduling for Hardware Accelerators
Open Access
- 9 October 2025
- journal article
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Embedded Computing Systems
- Vol. 24 (6) , 1-34
Abstract
Integrating domain-specific hardware accelerators on modern systems on chips (SoCs) has enabled complex applications, such as vision, natural language processing, autonomous driving, and augmented reality, on small form factors. This leads to challenges in the integration of accelerators, with high memory bandwidth requirements and strict deadlines, on the system’s memory hierarchy. The system-level shared cache, or last-level cache (LLC), is a critical resource shared by multi-core processors, GPUs, and hardware accelerators in modern heterogeneous SoCs. It significantly reduces the bottleneck at the off-chip memory and delivers high performance. With the integration of accelerators on the LLC gaining momentum, the on-chip shared cache management becomes vital. If not managed intelligently, the interference between cache requests from the cores and the accelerators can significantly deteriorate their performance. Given the architectural differences between DRAM and cache systems, the off-chip memory management strategies explored by previous works cannot be extended to the LLC. We propose a deadline-aware flexible LLC arbitration and scheduling framework, FLASH , to dynamically partition the LLC bandwidth between the accelerators and multi-core processors to meet the deadline given for the accelerator while minimizing the impact on the performance of the cores. FLASH arbitrates between the requests from the cores and the accelerators and schedules the requests depending on the accelerator’s progress and its chances of meeting the deadline. We evaluate FLASH across different workloads and hardware accelerator configurations to show that it not only achieves significantly better performance for the cores than other static scheduling policies but also significantly reduces the deadline miss rates of the accelerator.Keywords
Funding Information
- Semiconductor Research Corporation
- India Research Program (2020-IR-2979)
- R Systems Center of Excellence on Sustainable Artificial Intelligence
This publication has 39 references indexed in Scilit:
- Coupled Data Prefetch and Cache Partitioning Scheme for CPU-Accelerator System2023
- Xilinx Edge Processors2021
- Accelergy: An Architecture-Level Energy Estimation Methodology for Accelerator Designs2019
- Heterogeneous Cache Hierarchy Management for Integrated CPU-GPU Architecture2019
- Timeloop: A Systematic Approach to DNN Accelerator Evaluation2019
- Mastering the game of Go without human knowledgeNature, 2017
- Improving CPU Performance Through Dynamic GPU Access Throttling in CPU-GPU Heterogeneous Processors2017
- Going deeper with convolutions2015
- Aladdin: A pre-RTL, power-performance accelerator simulator enabling large design space exploration of customized architectures2014
- TAP: A TLP-aware cache management policy for a CPU-GPU heterogeneous architecture2012