MixedNet: Network Design Strategies for Cost-Effective Quantized CNNs
Open Access
- 23 August 2021
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 9, 117554-117564
Abstract
This paper proposes design strategies for a low-cost quantized neural network. To prevent the classification accuracy from being degraded by quantization, a structure-design strategy that utilizes a large number of channels rather than deep layers is proposed. In addition, a squeeze-and-excitation (SE) layer is adopted to enhance the performance of the quantized network. Through a quantitative analysis and simulations of the quantized key convolution layers of ResNet and MobileNets, a low-cost layer-design strategy for use when building a neural network is proposed. With this strategy, a low-cost network referred to as a MixedNet is constructed. A 4-bit quantized MixedNet example achieves an on-chip memory size reduction of 60% and fewer memory access by 53% with negligible classification accuracy degradation in comparison with conventional networks while also showing classification accuracy rates of approximately 73% for Cifar-100 and 93% for Cifar-10.Keywords
Funding Information
- Ministry of Science and ICT South Korea (IITP-2020-0-01847, NRF-2021R1C1C2009709)
This publication has 12 references indexed in Scilit:
- Deep-Learning-Based Joint Resource Scheduling Algorithms for Hybrid MEC NetworksIEEE Internet of Things Journal, 2019
- An Ultra-Low-Power Analog-Digital Hybrid CNN Face Recognition Processor Integrated with a CIS for Always-on Mobile Devices2019
- Squeeze-and-Excitation NetworksIEEE Transactions on Pattern Analysis and Machine Intelligence, 2019
- An Always-On 3.8 $\mu$ J/86% CIFAR-10 Mixed-Signal Binary CNN Processor With All Memory on Chip in 28-nm CMOSIEEE Journal of Solid-State Circuits, 2018
- MobileNetV2: Inverted Residuals and Linear Bottlenecks2018
- A Mixed-Signal Binarized Convolutional-Neural-Network Accelerator Integrating Dense Weight Storage and Multiplication for Reduced Data Movement2018
- In-Memory Computation of a Machine-Learning Classifier in a Standard 6T SRAM ArrayIEEE Journal of Solid-State Circuits, 2017
- 14.2 DNPU: An 8.1TOPS/W reconfigurable CNN-RNN processor for general-purpose deep neural networks2017
- Deep learningNature, 2015
- 90% write power-saving SRAM using sense-amplifying memory cellIEEE Journal of Solid-State Circuits, 2004