Automated APT Defense Using Reinforcement Learning and Attack Graph Risk-based Situation Awareness
Open Access
- 6 November 2023
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 23-33
Abstract
Advanced Persistent Threats pose significant risks to communication and infrastructure systems. While both heuristic and reinforcement learning have been applied to address this challenge, current approaches rely on fixed assumptions or use simplistic state representations and reward functions, compromising adaptability and accuracy. In this paper, we propose a novel intelligent agent system that enhances security against APTs by leveraging MulVAL attack graphs and quantitative risk assessment. Our approach translates complex MulVAL attack graph states into a compact RL state representation, enabling efficient learning in dynamic network environments. We integrate quantitative attack graph-based risk assessment into the RL framework, employing a hybrid reward function that incorporates residual risk, risk reduction, control efficacy, and cost. This risk-based approach provides enhanced context and situation awareness to the RL agent, facilitating more informed long-term decision-making. Using both Q-learning and Proximal Policy Optimization algorithms, we train and evaluate our system on an emulated industrial control system environment under realistic APT attacks. Experimental results demonstrate the significant benefits of our risk-based approach in guiding RL agents. Compared to non-risk configurations, our method shows improved success rates in APT mitigation, reduced control costs, which imply better long-term strategic planning.Keywords
Funding Information
- Bill & Melinda Gates Foundation (INV-001309)
- Dstl (PA0000000454)
- UKRI (EP/R007195/1 and EP/N510129/1)
This publication has 11 references indexed in Scilit:
- Good Night, and Good Luck: A Control Logic Injection Attack on OpenPLC2023
- A Model-Free Approach to Intrusion Response SystemsJournal of Information Security and Applications, 2022
- An integrated cyber security risk management framework and risk predication for the critical infrastructure protectionNeural Computing and Applications, 2022
- Enterprise Architecture Frameworks Assessment: Capabilities, Cyber Security and Resiliency Review2021
- An Analysis of LockerGoga Ransomware2019
- The application of the nanostructure aluminum in the blue organic light-emitting devicesOrganic Electronics, 2018
- Attack Graph-Based Countermeasure Selection Using a Stateful Return on Investment Metric2018
- Towards a Metamodel for SABSA Conceptual Architecture Descriptions2016
- Dynamical Calculation of Security Metrics for Countermeasure Selection in Computer Networks2016
- Analytical Approach to Attack Graph Analysis for Network Security2010