Analyzing Inference Privacy Risks Through Gradients In Machine Learning
Open Access
- 2 December 2024
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 3466-3480
Abstract
In distributed learning settings, models are iteratively updated with shared gradients computed from potentially sensitive user data. While previous work has studied various privacy risks of sharing gradients, our paper aims to provide a systematic approach to analyze private information leakage from gradients. We present a unified game-based framework that encompasses a broad range of attacks including attribute, property, distributional, and user disclosures. We investigate how different uncertainties of the adversary affect their inferential power via extensive experiments on five datasets across various data modalities. Our results demonstrate the inefficacy of solely relying on data aggregation to achieve privacy against inference attacks in distributed learning. We further evaluate five types of defenses, namely, gradient pruning, signed gradient descent, adversarial perturbations, variational information bottleneck, and differential privacy, under both static and adaptive adversary settings. We provide an information-theoretic view for analyzing the effectiveness of these defenses against inference from gradients. Finally, we introduce a method for auditing attribute inference privacy, improving the empirical estimation of worst-case privacy through crafting adversarial canary records.Keywords
All Related Versions
Funding Information
- National Institute of Health (U54HG012510)
This publication has 52 references indexed in Scilit:
- See through Gradients: Image Batch Recovery via GradInversion2021
- Soteria: Provable Defense against Privacy Leakage in Federated Learning from Representation Perspective2021
- Emotion recognition in low-resource settings: An evaluation of automatic feature selection methodsComputer Speech & Language, 2021
- The Secret Revealer: Generative Model-Inversion Attacks Against Deep Neural Networks2020
- Dreaming to Distill: Data-Free Knowledge Transfer via DeepInversion2020
- Age Progression/Regression by Conditional Adversarial Autoencoder2017
- Concentrated Differential Privacy: Simplifications, Extensions, and Lower Bounds2016
- A Methodology for Formalizing Model-Inversion Attacks2016
- Calibrating Noise to Sensitivity in Private Data Analysis2006
- A Simple Approach to Ordinal Classification2001