Real-Time Federated Evolutionary Neural Architecture Search
- 26 July 2021
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Evolutionary Computation
- Vol. 26 (2) , 364-378
Abstract
Federated learning is a distributed machine learning approach to privacy preservation and two major technical challenges prevent a wider application of federated learning. One is that federated learning raises high demands on communication resources, since a large number of model parameters must be transmitted between the server and clients. The other challenge is that training large machine learning models such as deep neural networks in federated learning requires a large amount of computational resources, which may be unrealistic for edge devices such as mobile phones. The problem becomes worse when deep neural architecture search is to be carried out in federated learning. To address the above challenges, we propose an evolutionary approach to real-time federated neural architecture search that not only optimizes the model performance but also reduces the local payload. During the search, a double-sampling technique is introduced, in which for each individual, only a randomly sampled sub-model is transmitted to a number of randomly sampled clients for training. This way, we effectively reduce computational and communication costs required for evolutionary optimization, making the proposed framework well suitable for real-time federated neural architecture search.Keywords
All Related Versions
This publication has 43 references indexed in Scilit:
- Deep Residual Learning for Image Recognition2016
- Rethinking the Inception Architecture for Computer Vision2016
- Taking the Human Out of the Loop: A Review of Bayesian OptimizationProceedings of the IEEE, 2015
- Privacy-Preserving Deep Learning2015
- A Knee Point-Driven Evolutionary Algorithm for Many-Objective OptimizationIEEE Transactions on Evolutionary Computation, 2014
- An Evolutionary Many-Objective Optimization Algorithm Using Reference-Point-Based Nondominated Sorting Approach, Part I: Solving Problems With Box ConstraintsIEEE Transactions on Evolutionary Computation, 2013
- Large-Scale Machine Learning with Stochastic Gradient Descent2010
- A Fast Elitist Non-dominated Sorting Genetic Algorithm for Multi-objective Optimization: NSGA-II2000
- Evolving artificial neural networksProceedings of the IEEE, 1999
- An evolutionary algorithm that constructs recurrent neural networksIEEE Transactions on Neural Networks, 1994