Rate‐Distortion Theory
Author:
- 15 April 2003
- book chapter
- Published by Wiley
Abstract
Rate‐distortion theoryis the branch of information theory that treats compressing the data produced by an information source down to a specified encoding rate that is strictly less than the source's entropy. This necessarily entails some lossiness, or distortion, between the original source data and the best approximation thereto that can be produced on the basis of the encoder's output bits. Rate‐distortion theory was introduced in the seminal works written in 1948 and 1959 byC. E. Shannon, the founder of information theory. We describe Shannon's contribution and then trace its subsequent development worldwide. Heavier than usual emphasis is placed on the concept of “matching” a channel to a source in the rate‐distortion sense, and also on the analogous matching of a source to a channel. Experimental evidence has been mounting in support of the hypothesis that living organisms often simultaneously achieve both of these matchings when processing their sensory inputs, thereby eliminating the need for the complex encoding and decoding operations that are needed in order to produce an information‐theoretically optimum system in the absence of such double matching.Keywords
This publication has 72 references indexed in Scilit:
- Context weighting for general finite-context sourcesIEEE Transactions on Information Theory, 1996
- Decomposition of Binary Random Fields and Zeros of Partition FunctionsThe Annals of Probability, 1987
- A Decomposition Theorem for Binary Markov Random FieldsThe Annals of Probability, 1987
- The rate-distortion region for multiple descriptions without excess rateIEEE Transactions on Information Theory, 1985
- Universal coding, information, prediction, and estimationIEEE Transactions on Information Theory, 1984
- Block Synchronization, Sliding-Block Coding, Invulnerable Sources and Zero Error Codes for Discrete Noisy ChannelsThe Annals of Probability, 1980
- Nonblock Source Coding with a Fidelity CriterionThe Annals of Probability, 1975
- Hiding and Covering in a Compact Metric SpaceThe Annals of Statistics, 1973
- Epsilon Entropy and Data CompressionThe Annals of Mathematical Statistics, 1971
- Bernoulli shifts with the same entropy are isomorphicAdvances in Mathematics, 1970