Tabular Interpolation Approach Based on Stable Random Projection for Estimating Empirical Entropy of High-Speed Network Traffic
Authors:
Open Access
- 27 September 2022
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 10, 104934-104953
Abstract
The empirical entropy of the network flow attributes is an essential measure for identifying anomalous network traffic. However, computing the exact entropy values for high-speed networks in real-time is computationally expensive. Accordingly, the present study replaces the complex computations of existing stable random projection methods for entropy estimation with a simple table lookup procedure. Notably, the size of the lookup table is reduced through a piece-wise linear interpolation heuristic in order to facilitate the implementation of the proposed scheme in resource-constrained pipeline environments. The proposed architecture enables entropy estimation to be performed using both the Log-Mean Estimator (LME) method and the New Estimator of Compressed Counting (NECC) algorithm reported in the literature. The feasibility of the proposed approach is verified empirically using both real-world network traffic traces and synthetic data streams. Moreover, the practical applicability is demonstrated via stream-based implementation in the programmable data planes of the NetFPGA-Plus framework and a Tofino P4 switch, respectively. The results indicate that the proposed tabulation-based entropy estimation scheme allows minimum-sized Ethernet frames to be processed with a wire speed of up to several hundred gigabits per second.Keywords
Funding Information
- Ministry of Science and Technology, Taiwan (108-2221-E-033-015, 109-2221-E-033-032-MY2)
This publication has 45 references indexed in Scilit:
- Detecting Distributed Denial of Service Attacks: Methods, Tools and Future DirectionsThe Computer Journal, 2013
- A near-optimal algorithm for estimating the entropy of a streamACM Transactions on Algorithms, 2010
- Data streaming algorithms for estimating entropy of network trafficACM SIGMETRICS Performance Evaluation Review, 2006
- Data Streams: Algorithms and ApplicationsFoundations and Trends® in Theoretical Computer Science, 2005
- Finding frequent items in data streamsTheoretical Computer Science, 2004
- On the Chambers-Mallows-Stuck method for simulating skewed stable random variablesStatistics & Probability Letters, 1996
- Probabilistic counting algorithms for data base applicationsJournal of Computer and System Sciences, 1985
- A Method for Simulating Stable Random VariablesJournal of the American Statistical Association, 1976
- New Methods in Statistical EconomicsJournal of Political Economy, 1963
- The Pareto-Levy Law and the Distribution of IncomeInternational Economic Review, 1960