ENASFL: A Federated Neural Architecture Search Scheme for Heterogeneous Deep Models in Distributed Edge Computing Systems
- 20 December 2023
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Network Science and Engineering
- Vol. 11 (3) , 2610-2620
Abstract
With the development of edge computing, deep neural networks (DNNs) are deployed in heterogeneous edge devices, which have considerable differences in equipment attributes. This makes it necessary to design and train model structures to ensure each deployment's accuracy, even when dealing with the same image recognition task. Such a huge project will consume a lot of human and computing resources. To automate the design, Neural Architecture Search (NAS) is proposed to explore the possibly efficient structure. To maintain parameters in situ, Federated Learning (FL) can train models cooperatively without disclosing the original data. However, combining NAS and FL is still in its infancy. This paper proposes an Edge-empowered Neural Architecture Search Scheme with Federated Learning (ENASFL), which can serve edge devices with different lightweight models. For the reusability of blocks in different models, our work searches for a multi-exit model, then selects the appropriate branch for each edge device. Further, we design a novel aggregation strategy for the cooperative training of these branches, which enables devices to work cooperatively and improve training efficiency. Experimental evaluations based on three real datasets demonstrate that the accuracy is improved by 3% on average when the model has compressed to $4\times$ the baseline network.Keywords
Funding Information
- National Natural Science Foundation of China (62072332)
- China NSFC (62002260, 62306208)
- Tianjin Xinchuang Haihe Lab of ITAI (22HHXCJC00002)
- Open Research Fund from Guangdong Laboratory of Artificial Intelligence and Digital Economy (GML-KF-22-03)
This publication has 32 references indexed in Scilit:
- Device scheduling and channel allocation for energy-efficient Federated Edge LearningComputer Communications, 2022
- Adaptive Early Exit of Computation for Energy-Efficient and Low-Latency Machine Learning over IoT Networks2022
- Effective Resource Allocation and Load Balancing in Hierarchical HetNets: Toward QoS-Aware Multi-Access Edge ComputingThe Computer Journal, 2021
- A survey on security and privacy of federated learningFuture Generation Computer Systems, 2021
- ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Design2018
- Densely Connected Convolutional Networks2017
- Aggregated Residual Transformations for Deep Neural Networks2017
- SGD: Saccharomyces Genome DatabaseNucleic Acids Research, 1998
- Simple statistical gradient-following algorithms for connectionist reinforcement learningMachine Learning, 1992
- On large deviations theory and asymptotically efficient Monte Carlo estimationIEEE Transactions on Information Theory, 1990