An Inter-Data Encoding Technique that Exploits Synchronized Data for Network Applications
- 10 September 2019
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Mobile Computing
- Vol. 20 (1) , 76-92
Abstract
In a variety of network applications, there exists significant amount of shared data between two end hosts. Examples include data synchronization services that replicate data from one node to another. Given that shared data may have high correlation with new data to transmit, we question how such shared data can be best utilized to improve the efficiency of data transmission. To answer this, we develop an inter-data encoding technique, SyncCoding, that effectively replaces bit sequences of the data to be transmitted with the pointers to their matching bit sequences in the shared data so called references. By doing so, SyncCoding can reduce data traffic, speed up data transmission, and save energy consumption for transmission. Our evaluations of SyncCoding implemented in Linux show that it outperforms existing popular encoding techniques, Brotli, LZMA, Deflate, and Deduplication. The gains of SyncCoding over those techniques in the perspective of data size after compression in a cloud storage scenario are about 12.5%, 20.8%, 30.1%, and 66.1%, and are about 78.4%, 80.3%, 84.3%, and 94.3% in a web browsing scenario, respectively.Keywords
Funding Information
- Institute for Information and Communications Technology Promotion (2015-0-00278)
- Research on Near-Zero Latency Network for 5G Immersive Service (2017-0-00692)
- National Science Foundation (CNS-1901057, CNS-1719371)
- Office of Naval Research (N00014-17-1-2417)
This publication has 24 references indexed in Scilit:
- WAN-optimized replication of backup datasets using stream-informed delta compressionACM Transactions on Storage, 2012
- A study of practical deduplicationACM Transactions on Storage, 2012
- The rijndael algorithmIEEE Potentials, 2004
- A low-bandwidth network file system2001
- A protocol-independent technique for eliminating redundant network trafficACM SIGCOMM Computer Communication Review, 2000
- The sliding-window Lempel-Ziv algorithm is asymptotically optimalProceedings of the IEEE, 1994
- High-speed VLSI designs for Lempel-Ziv-based data compressionIEEE Transactions on Circuits and Systems II: Express Briefs, 1993
- A universal algorithm for sequential data compressionIEEE Transactions on Information Theory, 1977
- On Information and SufficiencyThe Annals of Mathematical Statistics, 1951
- A Mathematical Theory of CommunicationBell System Technical Journal, 1948