Entropy-Driven Action Randomization: A Deployment-Time Defense for Environment Privacy in Deep Reinforcement Learning
Open Access
- 8 July 2026
- journal article
- research article
- Published by MDPI AG in Electronics
- Vol. 15 (14) , 2998
Abstract
Deep reinforcement learning (DRL) agents implicitly memorize the structure of their training environment, allowing an attacker to reconstruct it by querying the action outputs of a deployed model and causing environment-privacy leakage. To address this, this study proposes Entropy-Driven Action Randomization (EDAR), a deployment-time action randomization defense. Inspired by the exponential mechanism from differential privacy, EDAR replaces deterministic greedy action selection with probabilistic action sampling based on a normalized Q-value utility function, without altering the trained policy. A state-adaptive dynamic privacy budget is further designed, guided by the Shannon entropy of the action distribution. A simulated-annealing-based policy inversion attack is used to quantify privacy leakage, measured by the normalized recovery rate (NRR). Experiments on GridWorld environments ranging from 7 × 7 to 13 × 13 show that, at a privacy level comparable to a strong fixed budget, the proposed dynamic budget raises the reward retention from 6.2% to 78.9% while keeping a comparable NRR reduction. These results indicate that adapting the privacy budget to per-state policy determinism yields a markedly better privacy–utility trade-off than existing training-perturbation-based methods. We clarify that the differential privacy property invoked here is a per-query, mechanism-level guarantee of indistinguishability between neighboring observation states under a fixed trained model. The protection of the environment structure itself is established empirically through the policy inversion attack. Experiments are conducted on discrete GridWorld navigation tasks; the conclusions are scoped to small value-based navigation settings.Keywords
Funding Information
- Guizhou Provincial Science and Technology Department (QianKeHe Basic MS [2025] 622)
- Guizhou Provincial Science and Technology Department (QianKeHe Basic—[2024] Youth 126)
This publication has 18 references indexed in Scilit:
- Dynamic Event-Triggered Control for Human–Machine Cooperative Systems Based on Dynamic Authority AllocationIEEE Transactions on Systems, Man, and Cybernetics: Systems, 2026
- An innovative neural network architecture designed for industrial fault diagnosis with hierarchical adaptive attention mechanismProcess Safety and Environmental Protection, 2025
- Privacy preservation in deep reinforcement learning: A training perspectiveKnowledge-Based Systems, 2024
- Membership Inference Attacks From First Principles2022
- Privacy-Preserving in Double Deep-Q-Network with Differential Privacy in Continuous Spaces2022
- Challenges and Countermeasures for Adversarial Attacks on Deep Reinforcement LearningIEEE Transactions on Artificial Intelligence, 2021
- Membership Inference Attacks Against Machine Learning Models2017
- Human-level control through deep reinforcement learningNature, 2015
- Mechanism Design via Differential Privacy2007
- Optimization by Simulated AnnealingScience, 1983