GPNPU: Enabling Efficient Hardware-Based Direct Convolution with Multi-Precision Support in GPU Tensor Cores
Authors:
- 1 July 2020
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 1-6
Abstract
To tailor for DNN (Deep Neural Network) acceleration, GPU has migrated to new architectures such as NVIDIA Volta and Turing that incorporate dedicated Tensor Cores. Although good at GEMM (generic matrix-matrix multiplication), Tensor Cores still have inefficiency facing convolutions with certain layer structures. This paper proposes a GPNPU (General-Purpose Neural-network Processing Unit) architecture, which offers another option of direct convolution in GPU. It stitches the direct convolution dataflow into the Tensor Cores with little hardware support, and resorts to regulated data layout with stripe-mined convolution execution to achieve higher performance and power efficiency, while retaining the general programability as GPU. We further apply a unified core design to support varied operand types and precision for higher computing throughput. The evaluation shows that GPNPU can outperform Tensor Cores on typical DNNs by 1.4X for inference (FP16) and 1.2X for training with much reduced power. The INT8 performance even increases to 2.4X. Our study demonstrates that it is possible and appealing to refine the Tensor Cores for greater DNN acceleration, while conforming to GPU architecture for the programmability necessary in future DNN evolution.Keywords
This publication has 12 references indexed in Scilit:
- 2.9TOPS/W Reconfigurable Dense/Sparse Matrix-Multiply Accelerator with Unified INT8/INTI6/FP16 Datapath in 14NM Tri-Gate CMOS2018
- Bit Fusion: Bit-Level Dynamically Composable Architecture for Accelerating Deep Neural Network2018
- Efficient Processing of Deep Neural Networks: A Tutorial and SurveyProceedings of the IEEE, 2017
- In-Datacenter Performance Analysis of a Tensor Processing Unit2017
- Performance Analysis of GPU-Based Convolutional Neural Networks2016
- ShiDianNaoACM SIGARCH Computer Architecture News, 2015
- Optimizing FPGA-based Accelerator Design for Deep Convolutional Neural Networks2015
- A 240 G-ops/s Mobile Coprocessor for Deep Neural Networks2014
- 1.1 Computing's energy problem (and what we can do about it)2014
- A dynamically configurable coprocessor for convolutional neural networksACM SIGARCH Computer Architecture News, 2010