Dynamic Watermarking for Finite Markov Decision Processes
Open Access
- 6 January 2025
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Open Journal of Control Systems
- Vol. 4, 41-52
Abstract
Dynamic watermarking is an active intrusion detection technique that can potentially detect replay attacks, spoofing attacks, and deception attacks in the feedback channel for control systems. In this paper, we develop a novel dynamic watermarking algorithm for finite-state finite-action Markov decision processes. We derive a lower bound on the mean time between false alarms and an upper bound on the mean delay between the time an attack occurs and when it is detected. We further compute the sensitivity of the performance of the control system as a function of the watermark. We demonstrate the effectiveness of the proposed dynamic watermarking algorithm by detecting a spoofing attack in a sensor network system.Keywords
All Related Versions
Funding Information
- University Alliance Project by Ford Motor Company
This publication has 39 references indexed in Scilit:
- Detecting Deception Attacks on Autonomous Vehicles via Linear Time-Varying Dynamic Watermarking2020
- Control Systems Under Attack: The Securable and Unsecurable Subspaces of a Linear Stochastic SystemLecture Notes in Control and Information Sciences - Proceedings, 2018
- The securable subspace of a linear stochastic system with malicious sensors and actuators2017
- Quickest Change Detection2014
- Online change detection of Markov chains with unknown post-change transition probabilitiesCommunications in Statistics - Theory and Methods, 2013
- Secure control against replay attacks2009
- Lessons Learned from the Maroochy Water Breach2007
- From Perturbation Analysis to Markov Decision Processes and Reinforcement LearningDiscrete Event Dynamic Systems, 2003
- Hoeffding's inequality for uniformly ergodic Markov chainsStatistics & Probability Letters, 2002
- Recurrence conditions for Markov decision processes with Borel state space: A surveyAnnals of Operations Research, 1991