CAMP: A New Bitmap Index for Data Retrieval in Traffic Archival

Authors:

Abstract
Traffic archival, an indispensable task for network analysis, has suffered from a huge amount of data. These rapid growing data can exceed storage capacity and thwart real-time analysis. In order to take insight from Internet traffic, bitmap indexing is applied in this field. However, raw bitmap indexes can consume more space and longer delay for loading larger indexes into memory. This leads to the invention of various bitmap index compression algorithms to save storage and ensure query efficiency. This paper proposes a new algorithm called common affix merging with partition. By merging common affixes in a bit sequence, it saves more storage and conducts at least one order of magnitude faster bitwise operations when compared with ROARING, WAH, CONCISE, and COMPAX. In practice, besides bitwise operations, a query should also contain the operations of loading index files into memory. The experimental results demonstrate that the speedup is still considerable even taking loading time into consideration.
Funding Information
  • National Natural Science Foundation of China (61472200, 61233016)
  • Ministry of Science and Technology of China under National 973 Basic Research Program (2013CB228206)
  • State Grid R&D project “Research on the Architecture of Information Communication System for Internet of Energy” (SGRIXTKJ[2015]253)
  • National Training Program of Innovation and Entrepreneurship for Undergraduates (201510003049, 201510003B066)

This publication has 12 references indexed in Scilit: