Comparative Analysis of Unsupervised Machine Learning Algorithms for Anomaly Detection in Network Data
Authors:
- 15 November 2023
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 1-6
Abstract
The prevalence of cyberattacks has been increasing annually, emphasizing the need for data-driven approaches for the prevention and mitigation of attacks. Analysis of network data facilitates the acquisition of usage information, thereby enabling the detection of anomalous behavior that deviates from established patterns. The identification of such anomalies may indicate potential threats to the network and may assist security management systems in preempting and mitigating potential threats. The present study employs unsupervised machine learning algorithms with dimensionality reduction to detect anomalies in the NSL-KDD network dataset. The data is labeled although the real network datasets are unlabeled. We defined and applied a methodology to detect anomalies in the dataset. In the study, we used the following unsupervised algorithms: Local Outlier Factor (LOF), Elliptic Envelope (EE), SGD One Class SVM (SGD) and Isolation Forest (IF). For the kind of data we analysed, the unsupervised learning algorithm that obtained the highest AUC and F1-score evaluation rate was the Elliptic Envelope.Keywords
Funding Information
- CAPES (426701/2018-6)
- CNPq (436649/2018-7)
This publication has 15 references indexed in Scilit:
- Three-way confusion matrix for classification: A measure driven viewInformation Sciences, 2020
- Unsupervised machine learning for network-centric anomaly detection in IoT2019
- Evaluation of Machine Learning Algorithms for Anomaly Detection in Industrial Networks2019
- Anomaly Detection Methods for Categorical DataACM Computing Surveys, 2019
- Unsupervised Machine Learning for Networking: Techniques, Applications and Research ChallengesIEEE Access, 2019
- A detailed analysis of the KDD CUP 99 data set2009
- Isolation Forest2008
- Log correlation for intrusion detection: a proof of concept2004
- LOFACM SIGMOD Record, 2000
- A Fast Algorithm for the Minimum Covariance Determinant EstimatorTechnometrics, 1999