From DNNs to GANs: Review of efficient hardware architectures for deep learning
Author:
- Gaurab BhattacharyaGBGaurab Bhattacharya
Preprint
Abstract
In recent times, the trend in very large scale integration (VLSI) industry is multi-dimensional, for example, reduction of energy consumption, occupancy of less space, precise result, less power dissipation, faster response. To meet these needs, the hardware architecture should be reliable and robust to these problems. Recently, neural network and deep learning has been started to impact the present research paradigm significantly which consists of parameters in the order of millions, nonlinear function for activation, convolutional operation for feature extraction, regression for classification, generative adversarial networks. These operations involve huge calculation and memory overhead. Presently available DSP processors are incapable of performing these operations and they mostly face the problems, for example, memory overhead, performance drop and compromised accuracy. Moreover, if a huge silicon area is powered to accelerate the operation using parallel computation, the ICs will be having significant chance of burning out due to the considerable generation of heat. Hence, novel dark silicon constraint is developed to reduce the heat dissipation without sacrificing the accuracy. Similarly, different algorithms have been adapted to design a DSP processor compatible for fast performance in neural network, activation function, convolutional neural network and generative adversarial network. In this review, we illustrate the recent developments in hardware for accelerating the efficient implementation of deep learning networks with enhanced performance. The techniques investigated in this review are expected to direct future research challenges of hardware optimization for high-performance computations.Keywords
All Related Versions
This publication has 7 references indexed in Scilit:
- LACS: A High-Computational-Efficiency Accelerator for CNNsIEEE Access, 2019
- ApGAN: Approximate GAN for Robust Low Energy Learning From Imprecise ComponentsIEEE Transactions on Computers, 2019
- A Training-Efficient Hybrid-Structured Deep Neural Network With Reconfigurable Memristive SynapsesIEEE Transactions on Very Large Scale Integration (VLSI) Systems, 2019
- DNN Engine: A 28-nm Timing-Error Tolerant Sparse Deep Neural Network Processor for IoT ApplicationsIEEE Journal of Solid-State Circuits, 2018
- VLSI Implementation of Deep Neural Network Using Integral Stochastic ComputingIEEE Transactions on Very Large Scale Integration (VLSI) Systems, 2017
- GPUs and the Future of Parallel ComputingIEEE Micro, 2011
- Computing Models for FPGA-Based AcceleratorsComputing in Science & Engineering, 2008