Differentially Private Decentralized Optimization With Relay Communication
- 11 December 2024
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Information Forensics and Security
- Vol. 20 (15566013) , 724-736
Abstract
Security concerns in large-scale networked environments are becoming increasingly critical. To further improve the algorithm security from the design perspective of decentralized optimization algorithms, we introduce a new measure: Privacy Leakage Frequency (PLF), which reveals the relationship between communication and privacy leakage of algorithms, showing that lower PLF corresponds to lower privacy budgets. Based on such assertion, a novel differentially private decentralized primal–dual algorithm named DP-RECAL is proposed to take advantage of operator splitting method and relay communication mechanism to experience less PLF so as to reduce the overall privacy budget. To the best of our knowledge, compared with existing differentially private algorithms, DP-RECAL presents superior privacy performance and communication complexity. In addition, with uncoordinated network-independent stepsizes, we prove the convergence of DP-RECAL for general convex problems and establish a linear convergence rate under the metric subregularity. Evaluation analysis on least squares problem and numerical experiments on real-world datasets verify our theoretical results and demonstrate that DP-RECAL can defend some classical gradient leakage attacks.Keywords
All Related Versions
Funding Information
- National Natural Science Foundation of China (62176056, 62003084, 62473098)
- Young Elite Scientists Sponsorship Program by China Association for Science and Technology (62473098, 2021QNRC001)
This publication has 35 references indexed in Scilit:
- Differentially Private Federated Learning with Local Regularization and Sparsification2022
- See through Gradients: Image Batch Recovery via GradInversion2021
- Deep Leakage from Gradients2020
- Privacy-Preserving Distributed Learning with Nonsmooth Objective Functions2020
- Towards Decentralized Deep Learning with Differential Privacy2019
- A Review on Machine Learning Classification Techniques for Plant Disease Detection2019
- Recycled ADMM: Improve Privacy and Accuracy with Less Computation in Distributed Algorithms2018
- Asymmetric forward–backward–adjoint splitting for solving monotone inclusions involving three operatorsComputational Optimization and Applications, 2017
- Concentrated Differential Privacy: Simplifications, Extensions, and Lower Bounds2016
- Variational Analysis1998