DP-Norm: Differential Privacy Primal-Dual Algorithm for Decentralized Federated Learning
Open Access
- 18 April 2024
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Information Forensics and Security
- Vol. 19 (15566013) , 5783-5797
Abstract
A novel algorithm is proposed for highly privacy-preserving decentralized federated learning (FL). Several studies have reported security risks in decentralized FL by reconstructing data even from model update differences. A common approach to overcome this issue is to use the diffusion process following differential privacy (DP), i.e., message passing between nodes is hidden by noise. However, this often makes the learning process unstable, leading to degraded results compared to without using DP diffusion process. In this paper, we propose a primal-dual DP algorithm with denoising normalization (DP-Norm) for less sensitivity to noise/interference, such as DP diffusion and heterogeneous data allocation. For DP-Norm, privacy analysis to determine minimal noise level and convergence analysis are conducted. Through image classification benchmark tests, we confirmed that DP-Norm performed close to the single-node reference score, even when statistically heterogeneous data was allocated on six nodes.Keywords
This publication has 20 references indexed in Scilit:
- Training Speech Recognition Models with Federated Learning: A Quality/Cost Framework2021
- Feature Inference Attack on Model Predictions in Vertical Federated Learning2021
- Variance-Reduced Decentralized Stochastic Optimization With Accelerated ConvergenceIEEE Transactions on Signal Processing, 2020
- DP-ADMM: ADMM-Based Distributed Learning With Differential PrivacyIEEE Transactions on Information Forensics and Security, 2019
- Beyond Inferring Class Representatives: User-Level Privacy Leakage From Federated Learning2019
- Derivation and Analysis of the Primal-Dual Method of Multipliers Based on Monotone Operator TheoryIEEE Transactions on Signal and Information Processing over Networks, 2018
- The Composition Theorem for Differential PrivacyIEEE Transactions on Information Theory, 2017
- Distributed Optimization Using the Primal-Dual Method of MultipliersIEEE Transactions on Signal and Information Processing over Networks, 2017
- Deep Residual Learning for Image Recognition2016
- Diffusion Adaptation Strategies for Distributed Optimization and Learning Over NetworksIEEE Transactions on Signal Processing, 2012