Analysis of Autonomous Penetration Testing Through Reinforcement Learning and Recommender Systems
Authors:
- Ariadna Claudia MorenoAriadna Claudia Moreno
- Aldo Hernandez-SuarezAHAldo Hernandez-Suarez
- Gabriel Sanchez-PerezGSGabriel Sanchez-Perez
- Linda Karina Toscano-MedinaLTLinda Karina Toscano-Medina
- Hector Perez-MeanaHPHector Perez-Meana
- Jose Portillo-PortilloJPJose Portillo-Portillo
- Jesus Olivares-MercadoJOJesus Olivares-Mercado
- Luis Javier García VillalbaLuis Javier García Villalba
Open Access
Abstract
Conducting penetration testing (pentesting) in cybersecurity is a crucial turning point for identifying vulnerabilities within the framework of Information Technology (IT), where real malicious offensive behavior is simulated to identify potential weaknesses and strengthen preventive controls. Given the complexity of the tests, time constraints, and the specialized level of expertise required for pentesting, analysis and exploitation tools are commonly used. Although useful, these tools often introduce uncertainty in findings, resulting in high rates of false positives. To enhance the effectiveness of these tests, Machine Learning (ML) has been integrated, showing significant potential for identifying anomalies across various security areas through detailed detection of underlying malicious patterns. However, pentesting environments are unpredictable and intricate, requiring analysts to make extensive efforts to understand, explore, and exploit them. This study considers these challenges, proposing a recommendation system based on a context-rich, vocabulary-aware transformer capable of processing questions related to the target environment and offering responses based on necessary pentest batteries evaluated by a Reinforcement Learning (RL) estimator. This RL component assesses optimal attack strategies based on previously learned data and dynamically explores additional attack vectors. The system achieved an F1 score and an Exact Match rate over %, demonstrating its accuracy and effectiveness in selecting relevant pentesting strategies.
Keywords
This publication has 39 references indexed in Scilit:
- A survey of large language models for cyber threat detectionComputers & Security, 2024
- Wilcoxon Statistics in R: Signed-Rank Test and Rank-Sum Test2024
- A survey on model-based reinforcement learningScience China Information Sciences, 2024
- Reinforcement learning algorithms: A brief surveyExpert Systems with Applications, 2023
- Efficient hyperparameters optimization through model-based reinforcement learning with experience exploiting and meta-learningSoft Computing, 2023
- Hierarchical reinforcement learning for efficient and effective automated penetration testing of large networksJournal of Intelligent Information Systems, 2022
- Toward Human-in-the-Loop AI: Enhancing Deep Reinforcement Learning via Real-Time Human Guidance for Autonomous DrivingEngineering, 2022
- Challenges of real-world reinforcement learning: definitions, benchmarks and analysisMachine Learning, 2021
- A systematic review on overfitting control in shallow and deep neural networksArtificial Intelligence Review, 2021
- Transformer models for text-based emotion detection: a review of BERT-based approachesArtificial Intelligence Review, 2021