Local Decode and Update for Big Data Compression
- 11 June 2020
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Information Theory
- Vol. 66 (9) , 5790-5805
Abstract
This paper investigates data compression that simultaneously allows local decoding and local update. The main result is a universal compression scheme for memoryless sources with the following features. The rate can be made arbitrarily close to the entropy of the underlying source, contiguous fragments of the source can be recovered or updated by probing or modifying a number of codeword bits that is on average linear in the size of the fragment, and the overall encoding and decoding complexity is quasilinear in the blocklength of the source. In particular, the local decoding or update of a single message symbol can be performed by probing or modifying on average a constant number of codeword bits. This latter part improves over previous best known results for which local decodability or update efficiency grows logarithmically with blocklength.Keywords
All Related Versions
Funding Information
- Nokia Bell Labs France within the Framework Computation over Encoded Data with Applications to Large Scale Storage
This publication has 51 references indexed in Scilit:
- A public resource facilitating clinical use of genomesProceedings of the National Academy of Sciences, 2012
- Large-scale compression of genomic sequence databases with the Burrows–Wheeler transformBioinformatics, 2012
- Robust relative compression of genomes with random accessBioinformatics, 2011
- Computational solutions to large-scale data management and analysisNature Reviews Genetics, 2010
- Data structures and compression algorithms for genomic sequence dataBioinformatics, 2009
- Dynamic entropy-compressed sequences and full-text indexesACM Transactions on Algorithms, 2008
- A simple storage scheme for strings achieving entropy boundsTheoretical Computer Science, 2007
- Are Bitvectors Optimal?SIAM Journal on Computing, 2002
- Compression of individual sequences via variable-rate codingIEEE Transactions on Information Theory, 1978
- A universal algorithm for sequential data compressionIEEE Transactions on Information Theory, 1977