Revisiting differentially private linear regression: optimal and adaptive prediction & estimation in unbounded domain
Author:
Preprint
Abstract
We revisit the problem of linear regression under a differential privacy constraint. By consolidating existing pieces in the literature, we clarify the correct dependence of the feature, label and coefficient domain in the optimization error and estimation error, hence revealing the delicate price of differential privacy in statistical estimation and statistical learning. Moreover, we propose simple modifications of two existing DP algorithms: (a) posterior sampling, (b) sufficient statistics perturbation, and show that they can be upgraded into **adaptive** algorithms that are able to exploit data-dependent quantities and behave nearly optimally for every instance. Extensive experiments are conducted on both simulated data and real data, which conclude that both AdaOPS and AdaSSP outperform the existing techniques on nearly all 36 data sets that we test on.Keywords
All Related Versions
This publication has 6 references indexed in Scilit:
- Differentially Private Model Selection with Penalized and Constrained LikelihoodJournal of the Royal Statistical Society Series A: Statistics in Society, 2017
- Differential Privacy for Statistics: What we Know and What we Want to LearnJournal of Privacy and Confidentiality, 2010
- Gaussian model selectionJournal of the European Mathematical Society, 2001
- Adaptive estimation of a quadratic functional by model selectionThe Annals of Statistics, 2000
- Statistical Methods for the Social SciencesJournal of the American Statistical Association, 1998
- Regression Towards Mediocrity in Hereditary Stature.The Journal of the Anthropological Institute of Great Britain and Ireland, 1886