Loss-Privacy Tradeoff in Federated Edge Learning
- 23 March 2022
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Journal of Selected Topics in Signal Processing
- Vol. 16 (3) , 546-558
Abstract
Federated learning has been recognized as a promising scheme to tackle the privacy issues in multi-access edge computing through periodically uploading machine learning (ML) model updates instead of the original user data to the edge server. However, there still remains privacy leakage in such federated edge learning (FEL) systems since the model updates accessed by the server can be utilized to recover the original data. In this paper, we consider a personalized differential privacy based FEL scheme to alleviate the privacy leakage by adding different noise perturbations to the model updates of each edge device. Note that the noise perturbations may degrade the ML model performance, which is depicted by the global loss function. It is thus necessary to achieve a loss-privacy tradeoff in FEL by determining the noise scales and the numbers of local model updates. To address this challenge, we first derive the convergence upper bound of the global loss function as well as the closed-form privacy leakage from an adversarial perspective. We then propose a distributed mechanism in which the choices of noise scales and numbers of local model updates are optimized, even when the server is unaware of the personalized privacy preferences of different edge devices. Extensive theoretical analysis and numerical evaluations demonstrate the effectiveness of our proposed method in terms of privacy preservation and the global loss of the learned model.Keywords
Funding Information
- National Key R&D Project of China (2020YFB1807100)
- Beijing Natural Science Foundation (4222005, L212027)
- National Science Foundation (61931019, 61941101)
This publication has 29 references indexed in Scilit:
- Making Big Money from Small Sensors: Trading Time-Series Data under Pufferfish Privacy2019
- Joint Task Assignment, Transmission, and Computing Resource Allocation in Multilayer Mobile Edge Computing SystemsIEEE Internet of Things Journal, 2018
- Generalized Gaussian Mechanism for Differential PrivacyIEEE Transactions on Knowledge and Data Engineering, 2018
- The MNIST Database of Handwritten Digit Images for Machine Learning Research [Best of the Web]IEEE Signal Processing Magazine, 2012
- Stochastic Gradient Descent Tricks2012
- Generalized Nash Equilibrium ProblemsAnnals of Operations Research, 2009
- A tutorial on decomposition methods for network utility maximizationIEEE Journal on Selected Areas in Communications, 2006
- Convex Optimization2004
- Incremental Subgradient Methods for Nondifferentiable OptimizationSIAM Journal on Optimization, 2001
- Lagrange Duality Theory for Convex Control ProblemsSIAM Journal on Control and Optimization, 1976