Scalable Privacy-Preserving Distributed Learning
Authors:
- David FroelicherDFDavid Froelicher
- Juan R. Troncoso-PastorizaJTJuan R. Troncoso-Pastoriza
- Apostolos PyrgelisAPApostolos Pyrgelis
- Sinem SavSSSinem Sav
- Joao Sa SousaJSJoao Sa Sousa
- Jean-Philippe BossuatJBJean-Philippe Bossuat
- Jean-Pierre HubauxJHJean-Pierre Hubaux
Preprint
Abstract
In this paper, we address the problem of privacy-preserving distributed learning and evaluation of machine learning models by analyzing it in the widespread MapReduce abstraction that we extend with privacy constraints. Following this abstraction, we instantiate SPINDLE (Scalable Privacy-preservINg Distributed LEarning), an operational distributed system that supports the privacy-preserving training and evaluation of generalized linear models on distributed datasets. SPINDLE enables the efficient execution of distributed gradient descent while ensuring data and model confidentiality, as long as at least one of the data providers is honest-but-curious. The trained model is then used for oblivious predictions on confidential data. SPINDLE is able to efficiently perform demanding training tasks that require a high number of iterations on large input data with thousands of features, distributed among hundreds of data providers. It relies on a multiparty homomorphic encryption scheme to execute high-depth computations on encrypted data without significant overhead. It further leverages on its distributed construction and the packing capabilities of the cryptographic scheme to efficiently parallelize the computations at multiple levels. In our evaluation, SPINDLE performs the training of a logistic-regression model on a dataset of one million samples with 32 features distributed among 160 data providers in less than 176 seconds, yielding similar accuracy to non-secure centralized models.Keywords
All Related Versions
This publication has 25 references indexed in Scilit:
- Quantum supremacy using a programmable superconducting processorNature, 2019
- A Hybrid Approach to Privacy-Preserving Federated LearningInformatik-Spektrum, 2019
- A Hybrid of Dual and Meet-in-the-Middle Attack on Sparse and Ternary Secret LWEIEEE Access, 2019
- Cybersecurity in an Era with Quantum Computers: Will We Be Ready?IEEE Security & Privacy, 2018
- Secure genome-wide association analysis using multiparty computationNature Biotechnology, 2018
- SecureLR: Secure Logistic Regression Model via a Hybrid Cryptographic ProtocolIEEE/ACM Transactions on Computational Biology and Bioinformatics, 2018
- Privacy-Preserving Deep Learning via Additively Homomorphic EncryptionIEEE Transactions on Information Forensics and Security, 2017
- Rmind: A Tool for Cryptographically Secure Statistical AnalysisIEEE Transactions on Dependable and Secure Computing, 2016
- Big Data in medical research and EU data protection law: challenges to the consent or anonymise approachEuropean Journal of Human Genetics, 2015
- Smooth minimization of non-smooth functionsMathematical Programming, 2004