CryptGPU: Fast Privacy-Preserving Machine Learning on the GPU
Authors:
- 1 May 2021
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 1021-1038
Abstract
We introduce CryptGPU, a system for privacy-preserving machine learning that implements all operations on the GPU (graphics processing unit). Just as GPUs played a pivotal role in the success of modern deep learning, they are also essential for realizing scalable privacy-preserving deep learning. In this work, we start by introducing a new interface to losslessly embed cryptographic operations over secret-shared values (in a discrete domain) into floating-point operations that can be processed by highly-optimized CUDA kernels for linear algebra. We then identify a sequence of "GPU-friendly" cryptographic protocols to enable privacy-preserving evaluation of both linear and non-linear operations on the GPU. Our microbenchmarks indicate that our private GPU-based convolution protocol is over 150× faster than the analogous CPU-based protocol; for non-linear operations like the ReLU activation function, our GPU-based protocol is around 10× faster than its CPU analog. With CryptGPU, we support private inference and training on convolutional neural networks with over 60 million parameters as well as handle large datasets like ImageNet. Compared to the previous state-of-the-art, our protocols achieve a 2× to 8× improvement in private inference for large networks and datasets. For private training, we achieve a 6× to 36× improvement over prior state-of-the-art. Our work not only showcases the viability of performing secure multiparty computation (MPC) entirely on the GPU to newly enable fast privacy-preserving machine learning, but also highlights the importance of designing new MPC primitives that can take full advantage of the GPU’s computing capabilities.Keywords
All Related Versions
This publication has 26 references indexed in Scilit:
- High-Throughput Semi-Honest Secure Three-Party Computation with an Honest Majority2016
- Model Inversion Attacks that Exploit Confidence Information and Basic Countermeasures2015
- ImageNet Large Scale Visual Recognition ChallengeInternational Journal of Computer Vision, 2015
- Hacking smart machines with smarter ones: How to extract meaningful data from machine learning classifiersInternational Journal of Security and Networks, 2015
- ABY - A Framework for Efficient Mixed-Protocol Secure Two-Party Computation2015
- Deep, Big, Simple Neural Nets for Handwritten Digit RecognitionNeural Computation, 2010
- Security and Composition of Multiparty Cryptographic ProtocolsJournal of Cryptology, 2000
- Gradient-based learning applied to document recognitionProceedings of the IEEE, 1998
- Backpropagation Applied to Handwritten Zip Code RecognitionNeural Computation, 1989
- Secret sharing scheme realizing general access structureElectronics and Communications in Japan (Part III: Fundamental Electronic Science), 1989