Hardware-Aware Bayesian Neural Architecture Search of Quantized CNNs
- 26 July 2024
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Embedded Systems Letters
- Vol. 17 (1) , 42-45
Abstract
Advances in Neural Architecture Search (NAS) now provide a crucial assistance to design Hardware-efficient neural networks. This paper presents NAS for resource-efficient, weight-quantized Convolutional Neural Networks (CNNs), under computational complexity constraints (model size and number of computations). Bayesian Optimization is used to efficiently search for traceable CNN architectures within a continuous embedding space. This embedding is the latent space of a neural architecture autoencoder, regularized with a Maximum Mean Discrepancy penalization and a convex latent predictor of parameters. On CIFAR-100, and without quantization, we obtain 75% test accuracy with less than 2.5M parameters and 600M operations. NAS experiments on STL-10 with 32, 8 and 4 bit weights outperform some high-end architectures while enabling drastic model size reduction (6Mb to 840kb). It demonstrates our method’s ability to discover lightweight and high performing models, while showcasing the importance of quantization to improve the tradeoff between accuracy and model size.Keywords
This publication has 18 references indexed in Scilit:
- Efficient Search for Efficient Architecture2022
- Iterative Multiple-Path One-Shot NAS for the Optimized Performance2021
- Single Path One-Shot Neural Architecture Search with Uniform Sampling2020
- Rethinking Differentiable Search for Mixed-Precision Neural Networks2020
- APQ: Joint Search for Network Architecture, Pruning and Quantization Policy2020
- FBNet: Hardware-Aware Efficient ConvNet Design via Differentiable Neural Architecture Search2019
- MobileNetV2: Inverted Residuals and Linear Bottlenecks2018
- Learning Transferable Architectures for Scalable Image Recognition2018
- Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference2018
- Eyeriss: An Energy-Efficient Reconfigurable Accelerator for Deep Convolutional Neural NetworksIEEE Journal of Solid-State Circuits, 2016