The Variable-Increment Counting Bloom Filter
Authors:
- 7 August 2013
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE/ACM Transactions on Networking
- Vol. 22 (4) , 1092-1105
Abstract
Counting Bloom Filters (CBFs) are widely used in networking device algorithms. They implement fast set representations to support membership queries with limited error and support element deletions unlike Bloom Filters. However, they consume significant amounts of memory. In this paper, we introduce a new general method based on variable increments to improve the efficiency of CBFs and their variants. Unlike CBFs, at each element insertion, the hashed counters are incremented by a hashed variable increment instead of a unit increment. Then, to query an element, the exact value of a counter is considered and not just its positiveness. We present two simple schemes based on this method. We demonstrate that this method can always achieve a lower false positive rate and a lower overflow probability bound than CBF in practical systems. We also show how it can be easily implemented in hardware, with limited added complexity and memory overhead. We further explain how this method can extend many variants of CBF that have been published in the literature. We then suggest possible improvements of the presented schemes and provide lower bounds on their memory consumption. Lastly, using simulations with real-life traces and hash functions, we show how it can significantly improve the false positive rate of CBFs given the same amount of memory.Keywords
This publication has 14 references indexed in Scilit:
- The Bloom paradox: When not to use a Bloom filter?2012
- CassandraACM SIGOPS Operating Systems Review, 2010
- Counter braids2008
- Cache-, Hash- and Space-Efficient Bloom Filters2007
- Beyond bloom filters2006
- Spectral bloom filters2003
- Compressed Bloom filtersIEEE/ACM Transactions on Networking, 2002
- Summary cache: a scalable wide-area Web cache sharing protocolIEEE/ACM Transactions on Networking, 2000
- Bh[g]-Sequences with Large Upper DensityJournal of Number Theory, 1996
- Space/time trade-offs in hash coding with allowable errorsCommunications of the ACM, 1970