Empirical distribution of good channel codes with non-vanishing error probability (extended version)
Preprint
Abstract
This paper studies several properties of channel codes that approach the fundamental limits of a given (discrete or Gaussian) memoryless channel with a non-vanishing probability of error. The output distribution induced by an $\epsilon$-capacity-achieving code is shown to be close in a strong sense to the capacity achieving output distribution. Relying on the concentration of measure (isoperimetry) property enjoyed by the latter, it is shown that regular (Lipschitz) functions of channel outputs can be precisely estimated and turn out to be essentially non-random and independent of the actual code. It is also shown that the output distribution of a good code and the capacity achieving one cannot be distinguished with exponential reliability. The random process produced at the output of the channel is shown to satisfy the asymptotic equipartition property. Using related methods it is shown that quadratic forms and sums of $q$-th powers when evaluated at codewords of good AWGN codes approach the values obtained from a randomly generated Gaussian codeword.Keywords
All Related Versions
This publication has 17 references indexed in Scilit:
- Discrete isoperimetric and Poincaré-type inequalitiesProbability Theory and Related Fields, 1999
- Exponential Integrability and Transportation Cost Related to Logarithmic Sobolev InequalitiesJournal of Functional Analysis, 1999
- The empirical distribution of good codesIEEE Transactions on Information Theory, 1997
- Transportation cost for Gaussian and other product measuresGeometric and Functional Analysis, 1996
- Extremal properties of half-spaces for spherically invariant measuresJournal of Mathematical Sciences, 1978
- Every bad code has a good subcode: A local converse to the coding theoremProbability Theory and Related Fields, 1976
- Bounds on conditional probabilities with applications in multi-user communicationProbability Theory and Related Fields, 1976
- Asymptotic evaluation of certain markov process expectations for large time, ICommunications on Pure and Applied Mathematics, 1975
- On the Shannon capacity of an arbitrary channelIndagationes Mathematicae, 1974
- Optimal numberings and isoperimetric problems on graphsJournal of Combinatorial Theory, 1966