On Information Theoretic Fairness: Compressed Representations With Perfect Demographic Parity
Preprint
Abstract
In this article, we study the fundamental limits in the design of fair and/or private representations achieving perfect demographic parity and/or perfect privacy through the lens of information theory. More precisely, given some useful data $X$ that we wish to employ to solve a task $T$, we consider the design of a representation $Y$ that has no information of some sensitive attribute or secret $S$, that is, such that $I(Y;S) = 0$. We consider two scenarios. First, we consider a design desiderata where we want to maximize the information $I(Y;T)$ that the representation contains about the task, while constraining the level of compression (or encoding rate), that is, ensuring that $I(Y;X) \leq r$. Second, inspired by the Conditional Fairness Bottleneck problem, we consider a design desiderata where we want to maximize the information $I(Y;T|S)$ that the representation contains about the task which is not shared by the sensitive attribute or secret, while constraining the amount of irrelevant information, that is, ensuring that $I(Y;X|T,S) \leq r$. In both cases, we employ extended versions of the Functional Representation Lemma and the Strong Functional Representation Lemma and study the tightness of the obtained bounds. Every result here can also be interpreted as a coding with perfect privacy problem by considering the sensitive attribute as a secret.
Keywords
All Related Versions
This publication has 13 references indexed in Scilit:
- Classification Utility, Fairness, and Compactness via Tunable Information Bottleneck and Rényi MeasuresIEEE Transactions on Information Forensics and Security, 2023
- Data Disclosure With Non-Zero Leakage and Non-Invertible Leakage MatrixIEEE Transactions on Information Forensics and Security, 2021
- A Compression Perspective on Secrecy MeasuresIEEE Journal on Selected Areas in Information Theory, 2021
- On Perfect PrivacyIEEE Journal on Selected Areas in Information Theory, 2021
- A Design Framework for Strongly χ²-Private Data DisclosureIEEE Transactions on Information Forensics and Security, 2021
- Strong Functional Representation Lemma and Applications to Coding TheoremsIEEE Transactions on Information Theory, 2018
- Utility-Privacy Tradeoffs in Databases: An Information-Theoretic ApproachIEEE Transactions on Information Forensics and Security, 2013
- A new outlook on Shannon's information measuresIEEE Transactions on Information Theory, 1991
- A rate-distortion problem for a communication system with a secondary decoder to be hinderedIEEE Transactions on Information Theory, 1988
- A source coding problem for sources with additional outputs to keep secret from the receiver or wiretappers (Corresp.)IEEE Transactions on Information Theory, 1983