A Comparative Analysis of Self-Aware Reinforcement Learning Models for Real-Time Intrusion Detection in Fog Networks
Open Access
- 14 February 2026
- journal article
- research article
- Published by MDPI AG in Future Internet
- Vol. 18 (2) , 100
Abstract
Fog computing extends cloud services to the network edge, enabling low-latency processing for Internet of Things (IoT) applications. However, this distributed approach is vulnerable to a wide range of attacks, necessitating advanced intrusion detection systems (IDSs) that operate under resource constraints. This study proposes integrating self-awareness (online learning and concept drift adaptation) into a lightweight RL (reinforcement learning)-based IDS for fog networks and quantitatively comparing it with non-RL static thresholds and bandit-based approaches in real time. Novel self-aware reinforcement learning (RL) models, the Hierarchical Adaptive Thompson Sampling–Reinforcement Learning (HATS-RL) model, and the Federated Hierarchical Adaptive Thompson Sampling–Reinforcement Learning (F-HATS-RL), were proposed for real-time intrusion detection in a fog network. These self-aware RL policies integrated online uncertainty estimation and concept-drift detection to adapt to evolving attacks. The RL models were benchmarked against the static threshold (ST) model and a widely adopted linear bandit (Linear Upper Confidence Bound/LinUCB). A realistic fog network simulator with heterogeneous nodes and streaming traffic, including multi-type attack bursts and gradual concept drift, was established. The models’ detection performance was compared using metrics including latency, energy consumption, detection accuracy, and the area under the precision–recall curve (AUPR) and the area under the receiver operating characteristic curve (AUROC). Notably, the federated self-aware agent (F-HATS-RL) achieved the best AUROC (0.933) and AUPR (0.857), with a latency of 0.27 ms and the lowest energy consumption of 0.0137 mJ, indicating its ability to detect intrusions in fog networks with minimal energy. The findings suggest that self-aware RL agents can detect traffic–dynamic attack methods and adapt accordingly, resulting in more stable long-term performance. By contrast, a static model’s accuracy degrades under drift.Keywords
Funding Information
- Tshwane University of Technology
This publication has 37 references indexed in Scilit:
- Performance Evaluation of Reinforcement Learning-Based Intrusion Detection SystemsIEEE Access, 2025
- An adaptive collaborative intrusion detection system for vehicular fog computing networksEngineering Applications of Artificial Intelligence, 2025
- Novelty-aware concept drift detection for neural networksNeurocomputing, 2024
- Securing fog-assisted IoT smart homes: a federated learning-based intrusion detection approachCluster Computing, 2024
- Concept Drift–Based Intrusion Detection For Evolving Data Stream Classification In IDS: Approaches And Comparative StudyThe Computer Journal, 2024
- Concept Drift Detection in Data Stream Mining : A literature reviewJournal of King Saud University Computer and Information Sciences, 2022
- An Efficient Hybrid-DNN for DDoS Detection and Classification in Software-Defined IIoT NetworksIEEE Internet of Things Journal, 2022
- AS-IDS: Anomaly and Signature Based IDS for the Internet of ThingsJournal of Network and Systems Management, 2021
- Intrusion Detection Based on Autoencoder and Isolation Forest in Fog ComputingIEEE Access, 2020
- A survey of coordinated attacks and collaborative intrusion detectionComputers & Security, 2010