A Flexible Sketch-Based Network Traffic Monitoring Infrastructure
Open Access
- 10 July 2019
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 7, 92476-92498
Abstract
Sketch-based data streaming algorithms are used in many network traffic monitoring applications to obtain accurate estimates of traffic flow statistics. However, the current implementation of sketch-based methods in network monitoring is too application-specific. Their flexibility is limited as the hardware implementation of sketch data structure in the network device depends on the measurement tasks. The sketch counters only summarize flow statistics that are relevant to a specific measurement task and cannot be reused for different measurement tasks. In this paper, we propose FlexSketchMon, a system designed to provide the flexibility of using various sketch-based algorithms for traffic monitoring and measurement tasks. FlexSketchMon leverages on a novel data plane architecture that collects traffic flow statistics and provides arbitrary flow aggregations to the monitoring applications. The data plane design comprises a flow counter table and a flow key table for storing flow-level data. FlexSketchMon is implemented on the NetFPGA-SUME platform and is capable of processing network traffic at line rate in a worst-case scenario corresponding to a 64-byte minimum Ethernet frame size. The update of the flow counter table, which is the critical path in the proposed system, can achieve a throughput of 96 Gbps. The simulation results based on a real-world network traffic traces for three monitoring applications-estimation, superspreader detection, and heavy hitter detection-are presented to demonstrate the performance of FlexSketchMon. The results show that FlexSketchMon yields comparable and better measurement accuracy compared to previous approaches.Keywords
Funding Information
- Ministry of Science and Technology, Taiwan (MOST 107-2221-E-033-026, 106-2221-E-033-009, 105-2221-E-033-049)
This publication has 40 references indexed in Scilit:
- Hardware‐assisted estimation of entropy norm for high‐speed network trafficElectronics Letters, 2014
- NetFPGA SUME: Toward 100 Gbps as Research CommodityIEEE Micro, 2014
- A New Sketch Method for Measuring Host Connection Degree DistributionIEEE Transactions on Information Forensics and Security, 2014
- A survey of network flow applicationsJournal of Network and Computer Applications, 2013
- Summary Data Structures for Massive Data2013
- Tabulation-Based 5-Independent Hashing with Applications to Linear Probing and Second Moment EstimationSIAM Journal on Computing, 2012
- Linear Probing with 5-wise IndependenceSIAM Review, 2011
- An improved data stream summary: the count-min sketch and its applicationsJournal of Algorithms, 2005
- Cuckoo hashingJournal of Algorithms, 2004
- Finding frequent items in data streamsTheoretical Computer Science, 2004