On Sequences of Pairs of Dependent Random Variables
Author:
- H. S. WitsenhausenHWH. S. Witsenhausen
- 1 January 1975
- journal article
- Published by Society for Industrial & Applied Mathematics (SIAM) in SIAM Journal on Applied Mathematics
- Vol. 28 (1) , 100-113
Abstract
The generalized random variables $( {x,y} )$ have a given joint distribution. Pairs $( {x_i ,y_i } )$ are drawn independently. The observer of $( {x_1 , \cdots ,x_n } )$ and the observer of $( {y_1 , \cdots ,y_n } )$ each make a binary decision, of entropy bounded away from zero, with probability of disagreement $\varepsilon _n $. It is shown that $\varepsilon _n $ can be made to approach zero as $n \to \infty $ if and only if the maximum correlation of x and y is unity. Under a compactness condition, satisfied in particular when x and/or y takes only finitely many values, this occurs if and only if the joint distribution decomposes, that is when $\varepsilon _1 $ can be made to vanish by nontrivial decisions, as had been conjectured. Results are also obtained for nonidentically distributed pairs, for randomized decisions, for multivalued decisions and for decisions based on the infinite sequences. The question arose in the transmission of data from two dependent sources to two receivers. The results of Gács and Körner [1] for that problem are sharpened and clarified.
Keywords
This publication has 6 references indexed in Scilit:
- The common information of two dependent random variablesIEEE Transactions on Information Theory, 1975
- A theorem on the entropy of certain binary sequences and applications--IIIEEE Transactions on Information Theory, 1973
- On measures of dependenceActa Mathematica Hungarica, 1959
- New version of the probabilistic generalization of the large sieveActa Mathematica Hungarica, 1959
- Das statistische Problem der Korrelation als Variations‐ und Eigenwertproblem und sein Zusammenhang mit der AusgleichsrechnungZAMM - Journal of Applied Mathematics and Mechanics / Zeitschrift für Angewandte Mathematik und Mechanik, 1941
- A Connection between Correlation and ContingencyMathematical Proceedings of the Cambridge Philosophical Society, 1935