Neural architecture search for resource constrained hardware devices: A survey
Authors:
Open Access
- 3 July 2023
- journal article
- review article
- Published by Institution of Engineering and Technology (IET) in IET Cyber-Physical Systems: Theory & Applications
- Vol. 8 (3) , 149-159
Abstract
With the emergence of powerful and low‐energy Internet of Things devices, deep learning computing is increasingly applied to resource‐constrained edge devices. However, the mismatch between hardware devices with low computing capacity and the increasing complexity of Deep Neural Network models, as well as the growing real‐time requirements, bring challenges to the design and deployment of deep learning models. For example, autonomous driving technologies rely on real‐time object detection of the environment, which cannot tolerate the extra latency of sending data to the cloud, processing and then sending the results back to edge devices. Many studies aim to find innovative ways to reduce the size of deep learning models, the number of Floating‐point Operations per Second, and the time overhead of inference. Neural Architecture Search (NAS) makes it possible to automatically generate efficient neural network models. The authors summarise the existing NAS methods on resource‐constrained devices and categorise them according to single‐objective or multi‐objective optimisation. We review the search space, the search algorithm and the constraints of NAS on hardware devices. We also explore the challenges and open problems of hardware NAS.Keywords
Funding Information
- National Natural Science Foundation of China (62072076)
This publication has 24 references indexed in Scilit:
- Reinforcement learning for neural architecture search: A reviewImage and Vision Computing, 2019
- When Neural Architecture Search Meets Hardware Implementation: from Hardware Awareness to Co-Design2019
- Accuracy vs. Efficiency2019
- FBNet: Hardware-Aware Efficient ConvNet Design via Differentiable Neural Architecture Search2019
- MnasNet: Platform-Aware Neural Architecture Search for Mobile2019
- NAS-FPN: Learning Scalable Feature Pyramid Architecture for Object Detection2019
- Learning Transferable Architectures for Scalable Image Recognition2018
- MorphNet: Fast & Simple Resource-Constrained Structure Learning of Deep Networks2018
- Streaming Big Data Processing in Datacenter CloudsIEEE Cloud Computing, 2014
- Exponentiated Gradient versus Gradient Descent for Linear PredictorsInformation and Computation, 1997