Stream: A Modeling Framework for Fine-grained Layer Fusion on Multi-core DNN Accelerators
Authors:
- 1 April 2023
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 355-357
Abstract
To keep up with the ever-growing performance demand of DNN processing, specialized hardware (HW) accelerators are shifting towards multi-core architectures. Stream is the first open-source design space exploration (DSE) framework for co-optimization of HW architecture and fine-grained scheduling of such multi-core DNN accelerators. Stream supports finegrained layer fusion, to optimally trade-off energy, latency, and/or on-chip memory footprint for constrained edge devices. Validation against three SotA chips, together with a case study on seven HW architectures with different scheduling granularity, demonstrate the reliability and capabilities of Stream. Results show that high-level architectural decisions greatly impact HW efficiency under the fine-grained scheduling paradigm, reducing the energy-delay product from $2.4 \times$ for single-core architectures to up to $30 \times$ for heterogeneous multi-core architectures compared to traditional scheduling at layer granularity. Stream is open-source at github.com/ZigZag-Project/stream.
Keywords
This publication has 20 references indexed in Scilit:
- A Heterogeneous In-Memory Computing Cluster for Flexible End-to-End Inference of Real-World Deep Neural NetworksIEEE Journal on Emerging and Selected Topics in Circuits and Systems, 2022
- DIANA: An End-to-End Energy-Efficient Digital and ANAlog Hybrid Neural Network SoC2022
- Scalable and Programmable Neural Network Inference Accelerator Based on In-Memory ComputingIEEE Journal of Solid-State Circuits, 2021
- Heterogeneous Dataflow Accelerators for Multi-DNN Workloads2021
- Illusion of large on-chip memory by networked computing chips for neural network inferenceNature Electronics, 2021
- DLFusion: An Auto-Tuning Compiler for Layer Fusion on Deep Neural Network Accelerator2020
- Breaking High-Resolution CNN Bandwidth Barriers With Enhanced Depth-First ExecutionIEEE Journal on Emerging and Selected Topics in Circuits and Systems, 2019
- Timeloop: A Systematic Approach to DNN Accelerator Evaluation2019
- 14.5 Envision: A 0.26-to-10TOPS/W subword-parallel dynamic-voltage-accuracy-frequency-scalable Convolutional Neural Network processor in 28nm FDSOI2017
- Eyeriss: An Energy-Efficient Reconfigurable Accelerator for Deep Convolutional Neural NetworksIEEE Journal of Solid-State Circuits, 2016