A protocol-independent technique for eliminating redundant network traffic
Authors:
Open Access
- 28 August 2000
- journal article
- Published by Association for Computing Machinery (ACM) in ACM SIGCOMM Computer Communication Review
- Vol. 30 (4) , 87-95
Abstract
We present a technique for identifying repetitive information transfers and use it to analyze the redundancy of network traffic. Our insight is that dynamic content, streaming media and other traffic that is not caught by today's Web caches is nonetheless likely to derive from similar information. We have therefore adapted similarity detection techniques to the problem of designing a system to eliminate redundant transfers. We identify repeated byte ranges between packets to avoid retransmitting the redundant data. We find a high level of redundancy and are able to detect repetition that Web proxy caches are not. In our traces, after Web proxy caching has been applied, an additional 39% of the original volume of Web traffic is found to be redundant. Moreover, because our technique makes no assumptions about HTTP protocol syntax or caching semantics, it provides immediate benefits for other types of content, such as streaming media, FTP traffic, news and mail.Keywords
This publication has 6 references indexed in Scilit:
- On the scale and performance of cooperative Web proxy caching1999
- Performance of Web proxy caching in heterogeneous bandwidth environments1999
- Web proxy cachingACM SIGMETRICS Performance Evaluation Review, 1998
- Wide-area Internet traffic patterns and characteristicsIEEE Network, 1997
- WebExpress1996
- Compressing TCP/IP Headers for Low-Speed Serial Links1990