Bivariate measure of redundant information
- 23 January 2013
- journal article
- research article
- Published by American Physical Society (APS) in Physical Review E
- Vol. 87 (1) , 012130
Abstract
We define a measure of redundant information based on projections in the space of probability distributions. Redundant information between random variables is information that is shared between those variables. But, in contrast to mutual information, redundant information denotes information that is shared about the outcome of a third variable. Formalizing this concept, and being able to measure it, is required for the non-negative decomposition of mutual information into redundant and synergistic information. Previous attempts to formalize redundant or synergistic information struggle to capture some desired properties. We introduce a new formalism for redundant information and prove that it satisfies all the properties necessary outlined in earlier work, as well as an additional criterion that we propose to be necessary to capture redundancy. We also demonstrate the behavior of this new measure for several examples, compare it to previous measures, and apply it to the decomposition of transfer entropy.Keywords
All Related Versions
This publication has 17 references indexed in Scilit:
- Complexity measures from interaction structuresPhysical Review E, 2009
- Integrated Information in Discrete Dynamical Systems: Motivation and Theoretical FrameworkPLOS Computational Biology, 2008
- INFORMATION FLOWS IN CAUSAL NETWORKSAdvances in Complex Systems, 2008
- Gene Regulatory Network Reconstruction Using Conditional Mutual InformationEURASIP Journal on Bioinformatics and Systems Biology, 2008
- ARACNE: An Algorithm for the Reconstruction of Gene Regulatory Networks in a Mammalian Cellular ContextBMC Bioinformatics, 2006
- Synergy, Redundancy, and Independence in Population Codes, RevisitedThe Journal of Neuroscience, 2005
- Adaptive Rescaling Maximizes Information TransmissionNeuron, 2000
- How to measure the information gained from one symbolNetwork: Computation in Neural Systems, 1999
- Unconditionally secure key agreement and the intrinsic conditional informationIEEE Transactions on Information Theory, 1999
- A Mathematical Theory of CommunicationBell System Technical Journal, 1948