Scalable Privacy-Preserving Distributed Learning
Authors:
- David FroelicherDFDavid Froelicher
- Juan R. Troncoso-PastorizaJTJuan R. Troncoso-Pastoriza
- Apostolos PyrgelisAPApostolos Pyrgelis
- Sinem SavSSSinem Sav
- Joao Sa SousaJSJoao Sa Sousa
- Jean-Philippe BossuatJBJean-Philippe Bossuat
- Jean-Pierre HubauxJHJean-Pierre Hubaux
Preprint
Abstract
In this paper, we address the problem of privacy-preserving distributed learning and the evaluation of machine-learning models by analyzing it in the widespread MapReduce abstraction that we extend with privacy constraints. We design SPINDLE (Scalable Privacy-preservINg Distributed LEarning), the first distributed and privacy-preserving system that covers the complete ML workflow by enabling the execution of a cooperative gradient-descent and the evaluation of the obtained model and by preserving data and model confidentiality in a passive-adversary model with up to N-1 colluding parties. SPINDLE uses multiparty homomorphic encryption to execute parallel high-depth computations on encrypted data without significant overhead. We instantiate SPINDLE for the training and evaluation of generalized linear models on distributed datasets and show that it is able to accurately (on par with non-secure centrally-trained models) and efficiently (due to a multi-level parallelization of the computations) train models that require a high number of iterations on large input data with thousands of features, distributed among hundreds of data providers. For instance, it trains a logistic-regression model on a dataset of one million samples with 32 features distributed among 160 data providers in less than three minutes.Keywords
All Related Versions
This publication has 51 references indexed in Scilit:
- Deep Leakage from Gradients2020
- Concretely-Efficient Zero-Knowledge Arguments for Arithmetic Circuits and Their Application to Lattice-Based Cryptography2020
- Better Bootstrapping for Approximate Homomorphic Encryption2020
- Privacy-Preserving Federated Brain Tumour Segmentation2019
- Efficient Lattice-Based Zero-Knowledge Arguments with Standard Soundness: Construction and Applications2019
- Privacy-Preserving Multiparty Learning for Logistic Regression2018
- Privacy-Preserving Ridge Regression with only Linearly-Homomorphic Encryption2018
- Big Data and Machine Learning in Health CareJAMA, 2018
- Homomorphic Encryption for Arithmetic of Approximate Numbers2017
- Public-Key Cryptosystems Based on Composite Degree Residuosity Classes1999