Improving Availability of Vertical Federated Learning: Relaxing Inference on Non-overlapping Data
Open Access
- 28 June 2022
- journal article
- research article
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Intelligent Systems and Technology
- Vol. 13 (4) , 1-20
Abstract
Vertical Federated Learning (VFL) enables multiple parties to collaboratively train a machine learning model over vertically distributed datasets without data privacy leakage. However, there is a limitation of the current VFL solutions: current VFL models fail to conduct inference on non-overlapping samples during inference. This limitation seriously damages the VFL model’s availability because, in practice, overlapping samples may only take up a small portion of the whole data at each party which means a large part of inference tasks will fail. In this article, we propose a novel VFL framework which enables federated inference on non-overlapping data. Our framework regards the distributed features as privileged information which is available in the training period but disappears during inference. We distill the knowledge of such privileged features and transfer them to the parties’ local model which only processes local features. Furthermore, we adopt Oblivious Transfer (OT) to preserve data ID privacy during training and inference. Empirically, we evaluate the model on the real-world dataset collected from Criteo and Taobao. Besides, we also provide a security analysis of the proposed framework.Keywords
Funding Information
- Key-Area Research and Development Program of Guangdong Province (2021B0101400001)
- Hong Kong RGC (TRS T41-603/20-R, GRF 16213621, and GRF 16215119)
- Turing AI Computing Cloud
This publication has 26 references indexed in Scilit:
- Improved OT Extension for Transferring Short Secrets2013
- Practical Covertly Secure MPC for Dishonest Majority – Or: Breaking the SPDZ Limits2013
- A New Approach to Practical Active-Secure Two-Party Computation2012
- A new learning paradigm: Learning using privileged informationNeural Networks, 2009
- Privacy-Preserving Inter-database Operations2004
- Extending Oblivious Transfers Efficiently2003
- Non-Interactive Oblivious Transfer and Applications2001
- Public-Key Cryptosystems Based on Composite Degree Residuosity Classes1999
- Efficient Public-Key Cryptosystems Provably Secure Against Active Adversaries1999
- Precomputing Oblivious Transfer1995