Learning Near-Optimal Intrusion Responses Against Dynamic Attackers
Authors:
Preprint
Abstract
We study automated intrusion response and formulate the interaction between an attacker and a defender as an optimal stopping game where attack and defense strategies evolve through reinforcement learning and self-play. The game-theoretic modeling enables us to find defender strategies that are effective against a dynamic attacker, i.e. an attacker that adapts its strategy in response to the defender strategy. Further, the optimal stopping formulation allows us to prove that optimal strategies have threshold properties. To obtain near-optimal defender strategies, we develop Threshold Fictitious Self-Play (T-FP), a fictitious self-play algorithm that learns Nash equilibria through stochastic approximation. We show that T-FP outperforms a state-of-the-art algorithm for our use case. The experimental part of this investigation includes two systems: a simulation system where defender strategies are incrementally learned and an emulation system where statistics are collected that drive simulation runs and where learned strategies are evaluated. We argue that this approach can produce effective defender strategies for a practical IT infrastructure.Keywords
All Related Versions
This publication has 78 references indexed in Scilit:
- A Theory of Hypergames on Graphs for Synthesizing Dynamic Cyber Defense with Deception2021
- Using One-Sided Partially Observable Stochastic Games for Solving Zero-Sum Security Games with Sequential Attacks2020
- Partially Observable Stochastic Games for Cyber Deception Against Network Epidemic2020
- General Sum Markov Games for Strategic Detection of Advanced Persistent Threats Using Moving Target Defense in Cloud NetworksLecture Notes in Computer Science, 2019
- Control-Theoretic Approaches to Cyber-Security2019
- Game Theoretic Model of Strategic Honeypot Selection in Computer Networks2012
- A Jamming Game in Wireless Networks with Transmission Cost2007
- Dynkin games2006
- The problem of optimal stopping in a partially observable Markov chainJournal of Optimization Theory and Applications, 1985
- Estimates of Error Rates for Codes on Burst-Noise ChannelsBell System Technical Journal, 1963