Evolutionary Multiobjective Neural Architecture Search for Binary Neural Networks by Two-Stage Optimization
- 13 January 2026
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Cybernetics
- Vol. 56 (6) , 3210-3223
Abstract
Binary neural networks (BNNs) have been applied in limited resources and mobile devices because of their extreme model compression ability. However, manually designing suitable architectures is challenging given the specialized structure of binarized operations. Neural architecture search (NAS) provides a promising approach for designing high-performance BNN architectures. In practice, various situations require networks with different parameter sizes and performance levels. Therefore, this article proposes a multiobjective evolutionary NAS algorithm for BNNs based on a two-stage training strategy (MO-TS-BNAS) to solve these problems. First, the ApproxSign function is used to approximate the gradient error in the training of BNNs. To avoid the small model trap problem, two auxiliary objectives are introduced in nondominated sorting to retain larger models with similar errors. Then, a two-stage training strategy with flexible use of auxiliary objectives is proposed, forming the selection mechanism in environmental selection. The path dropout method is used in the second stage to prevent hypernetwork overfitting. In addition, the mini-batch gradient descent strategy is improved to speed up individual architecture evaluation and reduce time cost in the search process. Finally, the full precision baseline search space is binarized for general experimental comparison. Our MO-TS-BNAS algorithm balances the two different objective functions of the model size and error. A large number of experiments are carried out on the CIFAR10 and ImageNet datasets, and the results show the effectiveness of the proposed method.Keywords
Funding Information
- National Natural Science Foundation of China (62276202)
- Ministry of Education Joint Fund (8091B03072304)
- China Postdoctoral Science Foundation (2023T160501)
- Joint Funds of the Zhejiang Provincial Natural Science Foundation of China (LHZY24A010005)
- National Key Laboratory of Science and Technology on Communications (HTKJ2024KL504008)
This publication has 49 references indexed in Scilit:
- AdaBin: Improving Binary Neural Networks with Adaptive Binary Sets2022
- ReActNet: Towards Precise Binary Neural Network with Generalized Activation Functions2020
- BATS: Binary ArchitecTure Search2020
- NSGANetV2: Evolutionary Multi-objective Surrogate-Assisted Neural Architecture Search2020
- Single Path One-Shot Neural Architecture Search with Uniform Sampling2020
- Learning Architectures for Binary Networks2020
- Neural Architecture Search2019
- Bi-Real Net: Enhancing the Performance of 1-Bit CNNs with Improved Representational Capability and Advanced Training Algorithm2018
- XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networks2016
- Identity Mappings in Deep Residual Networks2016