A Distributed Computationally Aware Quantizer Design via Hyper Binning
Preprint
Abstract
We design a distributed function-aware quantization scheme for distributed functional compression. We consider $2$ correlated sources $X_1$ and $X_2$ and a destination that seeks an estimate $\hat{f}$ for the outcome of a continuous function $f(X_1,\,X_2)$. We develop a compression scheme called hyper binning in order to quantize $f$ via minimizing the entropy of joint source partitioning. Hyper binning is a natural generalization of Cover's random code construction for the asymptotically optimal Slepian-Wolf encoding scheme that makes use of orthogonal binning. The key idea behind this approach is to use linear discriminant analysis in order to characterize different source feature combinations. This scheme captures the correlation between the sources and the function's structure as a means of dimensionality reduction. We investigate the performance of hyper binning for different source distributions and identify which classes of sources entail more partitioning to achieve better function approximation. Our approach brings an information theory perspective to the traditional vector quantization technique from signal processing.
Keywords
All Related Versions
This publication has 38 references indexed in Scilit:
- Structured Sparsity Promoting FunctionsJournal of Optimization Theory and Applications, 2019
- Quantized Frame Expansions with ErasuresApplied and Computational Harmonic Analysis, 2001
- Partitioning the feature space of a classifier with linear hyperplanesIEEE Transactions on Speech and Audio Processing, 1999
- High-resolution source coding for non-difference distortion measures: multidimensional compandingIEEE Transactions on Information Theory, 1999
- Quantized overcomplete expansions in IR/sup N/: analysis, synthesis, and algorithmsIEEE Transactions on Information Theory, 1998
- Source coding and graph entropiesIEEE Transactions on Information Theory, 1996
- Lower bound on the mean-squared error in oversampled quantization of periodic signals using vector quantization analysisIEEE Transactions on Information Theory, 1996
- Finding parity in a simple broadcast networkIEEE Transactions on Information Theory, 1988
- The rate-distortion function for source coding with side information at the decoder\3-II: General sourcesInformation and Control, 1978
- On the representation of continuous functions of many variables by superposition of continuous functions of one variable and addition1963