Differentially Private Image Classification by Learning Priors from Random Processes
Open Access
- 31 March 2025
- journal article
- Published by Society for Privacy and Confidentiality Research in Journal of Privacy and Confidentiality
- Vol. 15 (1)
Abstract
In privacy-preserving machine learning, differentially private stochastic gradient descent (DP-SGD) performs worse than SGD due to per-sample gradient clipping and noise addition. A recent focus in private learning research is improving the performance of DP-SGD on private data by incorporating priors that are learned on real-world public data. In this work, we explore how we can improve the privacy-utility tradeoff of DP-SGD by learning priors from images generated by random processes and transferring these priors to private data. We propose DP-RandP, a three-phase approach. We attain new state-of-the-art accuracy when training from scratch on CIFAR10, CIFAR100, MedMNIST and ImageNet for a range of privacy budgets $\epsilon \in [1, 8]$. In particular, we improve the previous best reported accuracy on CIFAR10 from $60.6 \%$ to $72.3 \%$ for $\epsilon=1$.
Keywords
All Related Versions
This publication has 24 references indexed in Scilit:
- A Bias-Accuracy-Privacy Trilemma for Statistical EstimationJournal of the American Statistical Association, 2025
- MedMNIST v2 - A large-scale lightweight benchmark for 2D and 3D biomedical image classificationScientific Data, 2023
- Masked Autoencoders Are Scalable Vision Learners2022
- Pre-Training Without Natural ImagesInternational Journal of Computer Vision, 2022
- Analyzing and Improving the Image Quality of StyleGAN2020
- Momentum Contrast for Unsupervised Visual Representation Learning2020
- Augment Your Batch: Improving Generalization Through Instance Repetition2020
- Deep roto-translation scattering for object classification2015
- ImageNet: A large-scale hierarchical image database2009
- Occlusion Models for Natural Images: A Statistical Study of a Scale-Invariant Dead Leaves ModelInternational Journal of Computer Vision, 2001