On the Cross-Dataset Generalization of Machine Learning for Network Intrusion Detection
Preprint
Abstract
Network Intrusion Detection Systems (NIDS) are a fundamental tool in cybersecurity. Their ability to generalize across diverse networks is a critical factor in their effectiveness and a prerequisite for real-world applications. In this study, we conduct a comprehensive analysis on the generalization of machine-learning-based NIDS through an extensive experimentation in a cross-dataset framework. We employ four machine learning classifiers and utilize four datasets acquired from different networks: CIC-IDS-2017, CSE-CIC-IDS2018, LycoS-IDS2017, and LycoS-Unicas-IDS2018. Notably, the last dataset is a novel contribution, where we apply corrections based on LycoS-IDS2017 to the well-known CSE-CIC-IDS2018 dataset. The results show nearly perfect classification performance when the models are trained and tested on the same dataset. However, when training and testing the models in a cross-dataset fashion, the classification accuracy is largely commensurate with random chance except for a few combinations of attacks and datasets. We employ data visualization techniques in order to provide valuable insights on the patterns in the data. Our analysis unveils the presence of anomalies in the data that directly hinder the classifiers capability to generalize the learned knowledge to new scenarios. This study enhances our comprehension of the generalization capabilities of machine-learning-based NIDS, highlighting the significance of acknowledging data heterogeneity.Keywords
All Related Versions
This publication has 40 references indexed in Scilit:
- Cyber Threat Intelligence Sharing Scheme Based on Federated Learning for Network Intrusion DetectionJournal of Network and Systems Management, 2022
- Federated Learning for intrusion detection system: Concepts, challenges and future directionsComputer Communications, 2022
- Intrusion detection: A model based on the improved vision transformerTransactions on Emerging Telecommunications Technologies, 2022
- A two-stage intrusion detection system with auto-encoder and LSTMsApplied Soft Computing, 2022
- Towards a Standard Feature Set for Network Intrusion Detection System DatasetsMobile Networks and Applications, 2021
- Towards Model Generalization for Intrusion Detection: Unsupervised Machine Learning TechniquesJournal of Network and Systems Management, 2021
- An improved ensemble based intrusion detection technique usingXGBoostTransactions on Emerging Telecommunications Technologies, 2020
- Building an efficient intrusion detection system based on feature selection and ensemble classifierComputer Networks, 2020
- An efficient XGBoost–DNN-based classification model for network intrusion detection systemNeural Computing and Applications, 2020
- Unsupervised learning approach for network intrusion detection system using autoencodersThe Journal of Supercomputing, 2019