APRIL: Towards Scalable and Transferable Autonomous Penetration Testing in Large Action Space via Action Embedding
- 16 December 2024
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Dependable and Secure Computing
- Vol. 22 (3) , 2443-2459
Abstract
Penetration testing (pentesting) assesses cybersecurity through simulated attacks, while the conventional manual-based method is costly, time-consuming, and personnelconstrained. Reinforcement learning (RL) provides an agentenvironment interaction learning paradigm, making it a promising way for autonomous pentesting. However, agents' scalability in large action spaces and policy transferability across scenarios limit the applicability of RL-based autonomous pentesting. To address these challenges, we present a novel autonomous pentesting framework based on reinforcement learning (namely APRIL) to train agents that are scalable and transferable in large action spaces. In APRIL, we construct realistic, bounded, host-level state space via embedding techniques to avoid the complexities of dealing with unbounded network-level information. We employ semantic correlations between pentesting actions as prior knowledge to represent discrete action space into a continuous and semantically meaningful embedding space. Agents are then trained to reason over actions within the action embedding space, where two key methods are applied: an upper-confidence bound-based action refinement method to encourage efficient exploration, and a distance-aware loss to improve learning efficiency and generalization performance. We conduct experiments in simulated scenarios constructed based on virtualized vulnerable environments. The results demonstrate APRIL's scalability in large action spaces and its ability to facilitate policy transfer across diverse scenarios.Keywords
Funding Information
- National Key Research and Development Program of China (2021QY0500)
This publication has 25 references indexed in Scilit:
- Behaviour-diverse automatic penetration testing: a coverage-based deep reinforcement learning approachFrontiers of Computer Science, 2024
- Dense reinforcement learning for safety validation of autonomous vehiclesNature, 2023
- Modelling penetration testing with reinforcement learning using capture‐the‐flag challenges: Trade‐offs between model‐free learning and a priori knowledgeIET Information Security, 2023
- Securing critical infrastructures with a cybersecurity digital twinSoftware and Systems Modeling, 2023
- GAIL-PT: An intelligent penetration testing framework with generative adversarial imitation learningComputers & Security, 2022
- Autonomous navigation of UAV in multi-obstacle environments based on a Deep Reinforcement Learning approachApplied Soft Computing, 2022
- A survey of inverse reinforcement learning: Challenges, methods and progressArtificial Intelligence, 2021
- Automating post-exploitation with deep reinforcement learningComputers & Security, 2021
- Grandmaster level in StarCraft II using multi-agent reinforcement learningNature, 2019
- Human-level control through deep reinforcement learningNature, 2015