DeepTools: Compiler and Execution Runtime Extensions for RaPiD AI Accelerator
Authors:
- Swagath VenkataramaniSVSwagath Venkataramani
- Jungwook ChoiJCJungwook Choi
- Vijayalakshmi SrinivasanVSVijayalakshmi Srinivasan
- Wei WangWWWei Wang
- Jintao ZhangJZJintao Zhang
- Marcel SchaalMSMarcel Schaal
- Mauricio J. SerranoMSMauricio J. Serrano
- Kazuaki IshizakiKIKazuaki Ishizaki
- Hiroshi InoueHIHiroshi Inoue
- Eri OgawaEOEri Ogawa
- Moriyoshi OharaMOMoriyoshi Ohara
- Leland ChangLCLeland Chang
- 31 July 2019
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Micro
- Vol. 39 (5) , 102-111
Abstract
Systems specialized for Artificial Intelligence (AI) workloads are increasingly deployed across the spectrum of computing devices from edge to the cloud. However, their ubiquitous adoption requires bridging two seemingly conflicting challenges - the need to deliver extreme processing efficiencies while employing intuitive (and familiar) programming and user interfaces, making AI systems compelling even for common (non-expert) users. In this paper, we take a significant first step towards this goal and present an end-to-end software stack for the RAPID AI accelerator developed by IBM Research. We achieve this by developing a set of software extensions, called DEEPTOOLS, that leverage and work within popular deep learning frameworks (e.g., TensorFlow), thus requiring no additional user input while still enabling aggressive, accelerator-specific performance optimization that might otherwise be expected from a full, custom framework. DEEPTOOLS has two key components: (i) a compiler runtime called DEEPRT, which given a DNN graph automatically identifies how best to execute it on RAPID and constructs the requisite program binaries, (ii) an execution runtime called RAPIDLIB, which triggers and manages the execution of compute and data-transfer operations on RAPID. We demonstrate the utility of DEEPTOOLS by plugging it into TensorFlow and mapping multiple large and practical DNNs (AlexNet, VGG, ResNet, and LSTM) to RAPID. Further, our unique optimizations yielded substantial improvement in performance over hand-tuned homogeneous mappings.Keywords
This publication has 6 references indexed in Scilit:
- A Scalable Multi- TeraOPS Deep Learning Processor Core for AI Trainina and Inference2018
- A Configurable Cloud-Scale DNN Processor for Real-Time AI2018
- POSTER: Design Space Exploration for Performance Optimization of Deep Neural Networks on Shared Memory Accelerators2017
- ScaleDeep2017
- In-Datacenter Performance Analysis of a Tensor Processing Unit2017
- Eyeriss: An Energy-Efficient Reconfigurable Accelerator for Deep Convolutional Neural NetworksIEEE Journal of Solid-State Circuits, 2016