Improved Deep Recurrent Q-Network of POMDPs for Automated Penetration Testing
Open Access
- 14 October 2022
- journal article
- research article
- Published by MDPI AG in Applied Sciences
- Vol. 12 (20) , 10339
Abstract
With the development of technology, people’s daily lives are closely related to networks. The importance of cybersecurity protection draws global attention. Automated penetration testing is the novel method to protect the security of networks, which enhances efficiency and reduces costs compared with traditional manual penetration testing. Previous studies have provided many ways to obtain a better policy for penetration testing paths, but many studies are based on ideal penetration testing scenarios. In order to find potential vulnerabilities from the perspective of hackers in the real world, this paper models the process of black-box penetration testing as a Partially Observed Markov Decision Process (POMDP). In addition, we propose a new algorithm named , which is applied to the automated black-box penetration testing. In the POMDP model, an agent interacts with a network environment to choose a better policy without insider information about the target network, except for the start points. To handle this problem, we utilize a Long Short-Term Memory (LSTM) structure empowering agent to make decisions based on historical memory. In addition, this paper enhances the current algorithm using the structure of the neural network, the calculation method of the Q-value, and adding noise parameters to the neural network to advance the generalization and efficiency of this algorithm. In the last section, we conduct comparison experiments of the algorithm and other recent state-of-the-art (SOTA) algorithms. The experimental results vividly show that this novel algorithm is able to find a greater attack-path strategy for all vulnerable hosts in the automated black-box penetration testing. Additionally, the generalization and robustness of this algorithm are far superior to other SOTA algorithms in different size simulation scenarios based on the CyberBattleSim simulation developed by Microsoft.
Keywords
This publication has 12 references indexed in Scilit:
- GAIL-PT: An intelligent penetration testing framework with generative adversarial imitation learningComputers & Security, 2022
- Autonomous Penetration Testing Based on Improved Deep Q-NetworkApplied Sciences, 2021
- Simulating SQL injection vulnerability exploitation using Q-learning reinforcement learning agentsJournal of Information Security and Applications, 2021
- Autonomous Security Analysis and Penetration Testing2020
- NIG-AP: a new method for automated penetration testingFrontiers of Information Technology & Electronic Engineering, 2019
- Deep Reinforcement Learning with Double Q-LearningProceedings of the AAAI Conference on Artificial Intelligence, 2016
- Human-level control through deep reinforcement learningNature, 2015
- Reinforcement learning with limited reinforcement2008
- Solving Deep Memory POMDPs with Recurrent Policy Gradients2007
- Long Short-Term MemoryNeural Computation, 1997