Private Federated Learning with Autotuned Compression
Preprint
Abstract
We propose new techniques for reducing communication in private federated learning without the need for setting or tuning compression rates. Our on-the-fly methods automatically adjust the compression rate based on the error induced during training, while maintaining provable privacy guarantees through the use of secure aggregation and differential privacy. Our techniques are provably instance-optimal for mean estimation, meaning that they can adapt to the ``hardness of the problem" with minimal interactivity. We demonstrate the effectiveness of our approach on real-world datasets by achieving favorable compression rates without the need for tuning.Keywords
All Related Versions
This publication has 6 references indexed in Scilit:
- Two Models are Better Than One: Federated Learning is Not Private for Google GBoard Next Word Prediction2024
- Federated Learning of Gboard Language Models with Differential Privacy2023
- Robust Estimation via Robust Gradient EstimationJournal of the Royal Statistical Society Series B: Statistical Methodology, 2020
- Between Pure and Approximate Differential PrivacyJournal of Privacy and Confidentiality, 2017
- The Algorithmic Foundations of Differential Privacy2013
- Estimation des densités : Risque minimaxLecture Notes in Mathematics, 1978