A Survey on Statistical, Information, and Estimation—Theoretic Views on Privacy
- 27 August 2021
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE BITS the Information Theory Magazine
- Vol. 1 (1) , 45-56
Abstract
Privacy has become an emerging challenge in both information theory and computer science due to massive (centralized) collection of user data. In this article, we overview privacy-preserving mechanisms and metrics from the lenses of information theory, and unify different privacy metrics, including f-divergences, Rényi divergences, and differential privacy (DP), in terms of the probability likelihood ratio (and its logarithm). We review recent progress on the design of privacy-preserving mechanisms according to the privacy metrics in computer science, where DP is the standard privacy notion which controls the output shift given small input perturbation, and information theory, where the privacy is guaranteed by minimizing information leakage. In particular, for DP, we include its important variants (e.g., Rényi DP, Pufferfish privacy) and properties, discuss its connections with information-theoretic quantities, and provide the operational interpretations of its additive noise mechanisms. For information-theoretic privacy, we cover notable frameworks, including the privacy funnel, originated from rate-distortion theory and information bottleneck, to privacy guarantee against statistical inference/guessing, and information obfuscation on samples and features. Finally, we discuss the implementations of these privacy-preserving mechanisms in current data-driven machine learning scenarios, including deep learning, information obfuscation, federated learning, and dataset sharing.Keywords
Funding Information
- National Science Foundation (CIF 1900750, CIF CAREER 1845852)
- Amazon Research Award
- Google Research Faculty Award
- National Science Foundation
- National Institutes of Health
- DOD
This publication has 24 references indexed in Scilit:
- Deep Learning with Differential Privacy2016
- $f$ -Divergence InequalitiesIEEE Transactions on Information Theory, 2016
- Fundamental limits of perfect privacy2015
- PufferfishACM Transactions on Database Systems, 2014
- Social Media and Political Participation: Are Facebook, Twitter and YouTube Democratizing Our Political Systems?2011
- What Can We Learn Privately?SIAM Journal on Computing, 2011
- Privacy: Theory meets Practice on the Map2008
- Our Data, Ourselves: Privacy Via Distributed Noise Generation2006
- New directions in cryptographyIEEE Transactions on Information Theory, 1976
- The Wire-Tap ChannelBell System Technical Journal, 1975