Predictive Exit: Prediction of Fine-Grained Early Exits for Computation- and Energy-Efficient Inference
Open Access
- 26 June 2023
- journal article
- research article
- Published by Association for the Advancement of Artificial Intelligence (AAAI) in Proceedings of the AAAI Conference on Artificial Intelligence
- Vol. 37 (7) , 8657-8665
Abstract
By adding exiting layers to the deep learning networks, early exit can terminate the inference earlier with accurate results. However, the passive decision-making of whether to exit or continue the next layer has to go through every pre-placed exiting layer until it exits. In addition, it is hard to adjust the configurations of the computing platforms alongside the inference proceeds. By incorporating a low-cost prediction engine, we propose a Predictive Exit framework for computation- and energy-efficient deep learning applications. Predictive Exit can forecast where the network will exit (i.e., establish the number of remaining layers to finish the inference), which effectively reduces the network computation cost by exiting on time without running every pre-placed exiting layer. Moreover, according to the number of remaining layers, proper computing configurations (i.e., frequency and voltage) are selected to execute the network to further save energy. Extensive experimental results demonstrate that Predictive Exit achieves up to 96.2% computation reduction and 72.9% energy-saving compared with classic deep learning networks; and 12.8% computation reduction and 37.6% energy-saving compared with the early exit under state-of-the-art exiting strategies, given the same inference accuracy and latency.Keywords
All Related Versions
This publication has 9 references indexed in Scilit:
- Class-specific early exit design methodology for convolutional neural networksApplied Soft Computing, 2021
- EExNAS: Early-Exit Neural Architecture Search Solutions for Low-Power Wearable Devices2021
- Efficient adaptive inference for deep convolutional neural networks using hierarchical early exitsPattern Recognition, 2020
- Why Should We Add Early Exits to Neural Networks?Cognitive Computation, 2020
- Optimized training and scalable implementation of Conditional Deep Neural Networks with early exits for Fog-supported IoT applicationsInformation Sciences, 2020
- Edge AI: On-Demand Accelerating Deep Neural Network Inference via Edge ComputingIEEE Transactions on Wireless Communications, 2020
- Boomerang: On-Demand Cooperative Deep Neural Network Inference for Edge Intelligence on the Industrial Internet of ThingsIEEE Network, 2019
- Efficient Hardware Implementation of Cellular Neural Networks with Incremental Quantization and Early ExitACM Journal on Emerging Technologies in Computing Systems, 2018
- Energy-Efficient and Improved Image Recognition with Conditional Deep LearningACM Journal on Emerging Technologies in Computing Systems, 2017