Benchmark Accuracy Is Not Robustness: A Cross-Domain Study of ML-Based Intrusion Detection across IoT and Enterprise Traffic
Open Access
- 14 November 2026
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 109-120
Abstract
Machine learning-based intrusion detection systems often achieve strong performance on standard benchmarks, yet fail when deployed in a different environment. This paper investigates how well ML-based detectors transfer between IoT and enterprise traffic, using IoT-23 and UNSW-NB15 as two heterogeneous domains. Our experiments show that cross-domain performance drops sharply even after explicit feature alignment and limited target supervision, and that this drop is strongly asymmetric across transfer directions. Early detection experiments show that malicious signal is recoverable from partial traffic observations within a domain, but this property does not transfer equally between domains. The hybrid model that combines tabular, temporal, and prototype-based evidence provides interpretable insights into these failure modes, but does not consistently outperform simpler temporal baselines. Together our findings suggest that benchmark accuracy alone is a poor indicator of real-world robustness, and that intrusion detection system evaluation should place greater emphasis on cross-domain portability, data efficiency, and timeliness.This publication has 22 references indexed in Scilit:
- FORT-IDS: a federated, optimized, robust and trustworthy intrusion detection system for IIoT securityScientific Reports, 2025
- Transfer learning with XAI for robust malware and IoT network securityScientific Reports, 2025
- Explainable Cross-domain Evaluation of ML-based Network Intrusion Detection SystemsComputers and Electrical Engineering, 2023
- Survey on Enterprise Internet-of-Things systems (E-IoT): A security perspectiveAd Hoc Networks, 2022
- A collaborative approach to early detection of IoT BotnetComputers and Electrical Engineering, 2022
- Generative Deep Learning to Detect Cyberattacks for the IoT-23 DatasetIEEE Access, 2021
- Apply transfer learning to cybersecurity: Predicting exploitability of vulnerabilities by descriptionKnowledge-Based Systems, 2020
- DeepRan: Attention-based BiLSTM and CRF for Ransomware Early Detection and ClassificationInformation Systems Frontiers, 2020
- A novel approach for early malware detectionTransactions on Emerging Telecommunications Technologies, 2020
- Design of a Robust RF Fingerprint Generation and Classification Scheme for Practical Device Identification2019