Energy-Efficient DNN Training Processors on Micro-AI Systems
Open Access
- 2 November 2022
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Open Journal of the Solid-State Circuits Society
- Vol. 2, 259-275
Abstract
Many edge/mobile devices are now able to utilize deep neural networks (DNNs) thanks to the development of mobile DNN accelerators. Mobile DNN accelerators overcame the problems of limited computing resources and battery capacity by realizing energy-efficient inference. However, its passive behavior makes it difficult for DNN to provide active customization for individual users or its service environment. The importance of on-chip training is rising more and more to provide active interaction between DNN processors and ever-changing surroundings or conditions. Despite its advantages, the DNN training has more constraints than the inference such that it was considered impractical to be realized on mobile/edge devices. Recently, there are many trials to realize mobile DNN training, and a number of prior works will be summarized. Firstly, it arranges the new challenges of the DNN accelerator induced by training functionality and discusses new hardware features related to the challenges. Secondly, it explains algorithm-hardware co-optimization methods and explains why it becomes mainstream in mobile DNN training research. Thirdly, it compares the main differences between the conventional inference accelerators and recent training processors. Finally, the conclusion is made by proposing the future directions of the DNN training processor in micro-AI systems.Keywords
Funding Information
- Institute of Information & Communications Technology Planning & Evaluation (IITP) grant funded by the Korea Government (2022-0-01170)
This publication has 60 references indexed in Scilit:
- A Low-Power Convolutional Neural Network Face Recognition Processor and a CIS Integrated With Always-on Face DetectorIEEE Journal of Solid-State Circuits, 2017
- Efficient Processing of Deep Neural Networks: A Tutorial and SurveyProceedings of the IEEE, 2017
- A 21mW low-power recurrent neural network accelerator with quantization tables for embedded deep learning applications2017
- ESE2017
- 14.2 DNPU: An 8.1TOPS/W reconfigurable CNN-RNN processor for general-purpose deep neural networks2017
- Sparse Communication for Distributed Gradient Descent2017
- Eyeriss: An Energy-Efficient Reconfigurable Accelerator for Deep Convolutional Neural NetworksIEEE Journal of Solid-State Circuits, 2016
- Random synaptic feedback weights support error backpropagation for deep learningNature Communications, 2016
- EIEACM SIGARCH Computer Architecture News, 2016
- Learning representations by back-propagating errorsNature, 1986