Large-scale Secure XGB for Vertical Federated Learning
Authors:
Open Access
- 26 October 2021
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 443-452
Abstract
Privacy-preserving machine learning has drawn increasingly attention recently, especially with kinds of privacy regulations come into force. Under such situation, Federated Learning (FL) appears to facilitate privacy-preserving joint modeling among multiple parties. Although many federated algorithms have been extensively studied, there is still a lack of secure and practical gradient tree boosting models (e.g., XGB) in literature. In this paper, we aim to build large-scale secure XGB under vertically federated learning setting. We guarantee data privacy from three aspects. Specifically, (1) we employ secure multi-party computation techniques to avoid leaking intermediate information during training, (2) we store the output model in a distributed manner in order to minimize information release, and (3) we provide a novel algorithm for secure XGB predict with the distributed model. Furthermore, by proposing secure permutation protocols, we can improve the training efficiency and make the framework scale to large dataset. We conduct extensive experiments on both public datasets and real-world datasets, and the results demonstrate that our proposed XGB models provide not only competitive accuracy but also practical performance.Keywords
All Related Versions
This publication has 23 references indexed in Scilit:
- The TinyTable Protocol for 2-Party Secure Computation, or: Gate-Scrambling Revisited2017
- SecureML: A System for Scalable Privacy-Preserving Machine Learning2017
- Efficient and Private Scoring of Decision Trees, Support Vector Machines and Logistic Regression Models Based on Pre-ComputationIEEE Transactions on Dependable and Secure Computing, 2017
- XGBoost2016
- Privately Evaluating Decision Trees and Random ForestsProceedings on Privacy Enhancing Technologies, 2016
- ABY - A Framework for Efficient Mixed-Protocol Secure Two-Party Computation2015
- On the Power of Correlated Randomness in Secure Computation2013
- Secure Computation with Fixed-Point Numbers2010
- Oblivious transfer and polynomial evaluation1999
- How to generate and exchange secrets1986