Hyperdrive: A Systolically Scalable Binary-Weight CNN Inference Engine for mW IoT End-Nodes
Authors:
Preprint
Abstract
Deep neural networks have achieved impressive results in computer vision and machine learning. Unfortunately, state-of-the-art networks are extremely compute- and memory-intensive which makes them unsuitable for mW-devices such as IoT end-nodes. Aggressive quantization of these networks dramatically reduces the computation and memory footprint. Binary-weight neural networks (BWNs) follow this trend, pushing weight quantization to the limit. Hardware accelerators for BWNs presented up to now have focused on core efficiency, disregarding I/O bandwidth and system-level efficiency that are crucial for deployment of accelerators in ultra-low power devices. We present Hyperdrive: a BWN accelerator dramatically reducing the I/O bandwidth exploiting a novel binary-weight streaming approach, and capable of handling high-resolution images by virtue of its systolic-scalable architecture. We achieve a 5.9 TOp/s/W system-level efficiency (i.e. including I/Os)---2.2x higher than state-of-the-art BNN accelerators, even if our core uses resource-intensive FP16 arithmetic for increased robustness.Keywords
All Related Versions
This publication has 13 references indexed in Scilit:
- NullHop: A Flexible Convolutional Neural Network Accelerator Based on Sparse Representations of Feature MapsIEEE Transactions on Neural Networks and Learning Systems, 2018
- XNORBIN: A 95 TOp/s/W hardware accelerator for binary convolutional neural networks2018
- Efficient Processing of Deep Neural Networks: A Tutorial and SurveyProceedings of the IEEE, 2017
- An IoT Endpoint System-on-Chip for Secure and Energy-Efficient Near-Sensor AnalyticsIEEE Transactions on Circuits and Systems I: Regular Papers, 2017
- YodaNN: An Architecture for Ultralow Power Binary-Weight CNN AccelerationIEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, 2017
- Near-Threshold RISC-V Core With DSP Extensions for Scalable IoT Endpoint DevicesIEEE Transactions on Very Large Scale Integration (VLSI) Systems, 2017
- Latency Critical IoT Applications in 5G: Perspective on the Design of Radio Interface and Network ArchitectureIEEE Communications Magazine, 2017
- Origami: A 803-GOp/s/W Convolutional Network AcceleratorIEEE Transactions on Circuits and Systems for Video Technology, 2016
- Faster R-CNN: Towards Real-Time Object Detection with Region Proposal NetworksIEEE Transactions on Pattern Analysis and Machine Intelligence, 2016
- Near-Threshold Computing: Reclaiming Moore's Law Through Energy Efficient Integrated CircuitsProceedings of the IEEE, 2010