On Zero-Delay Source-Channel Coding
Authors:
- 3 October 2014
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Information Theory
- Vol. 60 (12) , 7473-7489
Abstract
This paper studies the zero-delay source-channel coding problem, and specifically the problem of obtaining the vector transformations that optimally map between the m-dimensional source space and k-dimensional channel space, under a given transmission power constraint and for the mean square error distortion. The functional properties of the cost are studied and the necessary conditions for the optimality of the encoder and decoder mappings are derived. An optimization algorithm that imposes these conditions iteratively, in conjunction with the noisy channel relaxation method to mitigate poor local minima, is proposed. The numerical results show strict improvement over prior methods. The numerical approach is extended to the scenario of source-channel coding with decoder side information. The resulting encoding mappings are shown to be continuous relatives of, and in fact subsume as special case, the Wyner-Ziv mappings encountered in digital distributed source coding systems. A well-known result in information theory pertains to the linearity of optimal encoding and decoding mappings in the scalar Gaussian source and channel setting, at all channel signal-to-noise ratios (CSNRs). In this paper, the linearity of optimal coding, beyond the Gaussian source and channel, is considered and the necessary and sufficient condition for linearity of optimal mappings, given a noise (or source) distribution, and a specified a total power constraint are derived. It is shown that the Gaussian source-channel pair is unique in the sense that it is the only source-channel pair for which the optimal mappings are linear at more than one CSNR values. Moreover, the asymptotic linearity of optimal mappings is shown for low CSNR if the channel is Gaussian regardless of the source and, at the other extreme, for high CSNR if the source is Gaussian, regardless of the channel. The extension to the vector settings is also considered where besides the conditions inherited from the scalar case, additional constraints must be satisfied to ensure linearity of the optimal mappings.Keywords
All Related Versions
Funding Information
- National Science Foundation through the Division of Computing and Communication Foundations (CCF-0728986, CCF-1016861, CCF-1118075)
This publication has 50 references indexed in Scilit:
- On Optimal Jamming Over an Additive Noise Channel2013
- Data-Processing Bounds for Scalar Lossy Source Codes With Side Information at the DecoderIEEE Transactions on Information Theory, 2013
- Bounds for joint source-channel coding at high SNR2011
- Distortion lower bounds for finite dimensional joint source-channel coding2008
- Universal Zero-Delay Joint Source–Channel CodingIEEE Transactions on Information Theory, 2006
- Convex Optimization2004
- Elements of Information Theory2001
- The rate-distortion function for source coding with side information at the decoderIEEE Transactions on Information Theory, 1976
- Noiseless coding of correlated information sourcesIEEE Transactions on Information Theory, 1973
- A Counterexample in Stochastic Optimum ControlSIAM Journal on Control, 1968