Joint Source-Channel Coding Over Additive Noise Analog Channels Using Mixture of Variational Autoencoders
- 10 May 2021
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Journal on Selected Areas in Communications
- Vol. 39 (7) , 2000-2013
Abstract
In this paper, we present a learning scheme for Joint Source-Channel Coding (JSCC) over analog independent additive noise channels. We formulate the learning problem by showing that the minimization loss function from rate-distortion theory, is upper bounded by the loss function of the Variational Autoencoder (VAE). We show that when the source dimension is greater than the channel dimension, the encoding of two source samples in the neighborhood of each other need not be near each other. Such discontinuous projection needs to be accounted for by using multiple encoders and selecting an encoder to encode samples on a particular side of the discontinuity. We explore two selection methodologies, one based on an intuitive rule and the other where it is posed as a learning task in a Mixture-of-Experts (MoE) setup. We analyze the gradients of these methods and reason why the latter is better at avoiding local optima. We show the efficacy of the proposed methodology by simulating the performance of the system for JSCC of Gaussian sources over AWGN channels and showing that the learned solutions are close to or better than the ones proposed earlier. The proposed methodology is also naturally capable of generalizing to other source distributions which we showcase by simulating for Laplace sources. The learned systems are also robust to changes in channel conditions. Further, a single system can be trained to generalize over a range of channel conditions provided the channel conditions are known at both the transmitter and the receiver. Finally, we evaluate our proposed methodology on three different image datasets and showcase consistent improvement over existing methods due to the VAE formulation.Keywords
Funding Information
- National Science Foundation (ID MLWiNS-2003002)
- Intel Company
This publication has 32 references indexed in Scilit:
- On Zero-Delay Source-Channel CodingIEEE Transactions on Information Theory, 2014
- Power Constrained Channel Optimized Vector Quantizers used for Bandwidth Expansion2007
- Dimension Reducing Mappings in Joint Source-Channel Coding2006
- Predicting the exchange traded fund DIA with a combination of genetic algorithms and neural networksExpert Systems with Applications, 2004
- Bandwidth compression for continuous amplitude channels based on vector approximation to a continuous subset of the source signal space2002
- Joint design of block source codes and modulation signal setsIEEE Transactions on Information Theory, 1992
- Adaptive Mixtures of Local ExpertsNeural Computation, 1991
- Performance bounds and optimal linear coding for discrete-time multichannel communication systems (Corresp.)IEEE Transactions on Information Theory, 1980
- Theoretical limitations on the transmission of data from analog sourcesIEEE Transactions on Information Theory, 1965
- A Mathematical Theory of CommunicationBell System Technical Journal, 1948