STONNE: Enabling Cycle-Level Microarchitectural Simulation for DNN Inference Accelerators
Authors:
- 1 November 2021
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 201-213
Abstract
The design of specialized architectures for accelerating the inference procedure of Deep Neural Networks (DNNs) is a booming area of research nowadays. While first-generation rigid accelerator proposals used simple fixed dataflows tailored for dense DNNs, more recent architectures have argued for flexibility to efficiently support a wide variety of layer types, dimensions, and sparsity. As the complexity of these accelerators grows, the analytical models currently being used for design-space exploration are unable to capture execution-time subtleties, leading to inexact results in many cases as we demonstrate. This opens up a need for cycle-level simulation tools to allow for fast and accurate design-space exploration of DNN accelerators, and rapid quantification of the efficacy of architectural enhancements during the early stages of a design. To this end, we present STONNE (Simulation TOol of Neural Network/Engines), a cycle-level microarchitectural simulation framework that can plug into any high-level DNN framework as an accelerator device and perform full-model evaluation (i.e. we are able to simulate real, complete, unmodified DNN models) of state-of-the-art rigid and flexible DNN accelerators, both with and without sparsity support. As a proof of concept, we use STONNE in three use cases: i) a direct comparison of three dominant inference accelerators using real DNN models; ii) back-end extensions and iii) front-end extensions of the simulator to showcase the capability of STONNE to rapidly and precisely evaluate data-dependent optimizations.Keywords
Funding Information
- NSF (1909900)
This publication has 26 references indexed in Scilit:
- FlexFlow: A Flexible Dataflow Accelerator Architecture for Convolutional Neural Networks2017
- Eyeriss: An Energy-Efficient Reconfigurable Accelerator for Deep Convolutional Neural NetworksIEEE Journal of Solid-State Circuits, 2016
- DaDianNao: A Neural Network SupercomputerIEEE Transactions on Computers, 2016
- SQuAD: 100,000+ Questions for Machine Comprehension of Text2016
- ShiDianNao2015
- ImageNet Large Scale Visual Recognition ChallengeInternational Journal of Computer Vision, 2015
- Sniper2011
- The gem5 simulatorACM SIGARCH Computer Architecture News, 2011
- SimpleScalar: an infrastructure for computer system modelingComputer, 2002
- Rsim: simulating shared-memory multiprocessors with ILP processorsComputer, 2002