Learning Near-Optimal Intrusion Responses Against Dynamic Attackers
Open Access
- 7 July 2023
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Network and Service Management
- Vol. 21 (1) , 1158-1177
Abstract
We study automated intrusion response and formulate the interaction between an attacker and a defender as an optimal stopping game where attack and defense strategies evolve through reinforcement learning and self-play. The gametheoretic modeling enables us to find defender strategies that are effective against a dynamic attacker, i.e. an attacker that adapts its strategy in response to the defender strategy. Further, the optimal stopping formulation allows us to prove that best response strategies have threshold properties. To obtain nearoptimal defender strategies, we develop Threshold Fictitious Self-Play (T-FP), a fictitious self-play algorithm that learns Nash equilibria through stochastic approximation. We show that T-FP outperforms a state-of-the-art algorithm for our use case. The experimental part of this investigation includes two systems: a simulation system where defender strategies are incrementally learned and an emulation system where statistics are collected that drive simulation runs and where learned strategies are evaluated. We argue that this approach can produce effective defender strategies for a practical IT infrastructure.Keywords
All Related Versions
This publication has 137 references indexed in Scilit:
- A Theory of Hypergames on Graphs for Synthesizing Dynamic Cyber Defense with Deception2021
- General Sum Markov Games for Strategic Detection of Advanced Persistent Threats Using Moving Target Defense in Cloud NetworksLecture Notes in Computer Science, 2019
- Adversarial Attacks on Continuous Authentication Security: A Dynamic Game Approach2019
- Control-Theoretic Approaches to Cyber-Security2019
- Adversarial and Uncertain Reasoning for Adaptive Cyber DefenseLecture Notes in Computer Science, 2019
- Autonomous Cyber Deception2019
- Reinforcement Learning for Autonomous Defence in Software-Defined Networking2018
- Wiley Series in Probability and Statistics2008
- Feedback Control of Computing Systems2004
- Estimates of Error Rates for Codes on Burst-Noise ChannelsBell System Technical Journal, 1963