Abstract
Deep reinforcement learning (DRL) agents implicitly memorize the structure of their training environment, allowing an attacker to reconstruct it by querying the action outputs of a deployed model and causing environment-privacy leakage. To address this, this study proposes Entropy-Driven Action Randomization (EDAR), a deployment-time action randomization defense. Inspired by the exponential mechanism from differential privacy, EDAR replaces deterministic greedy action selection with probabilistic action sampling based on a normalized Q-value utility function, without altering the trained policy. A state-adaptive dynamic privacy budget is further designed, guided by the Shannon entropy of the action distribution. A simulated-annealing-based policy inversion attack is used to quantify privacy leakage, measured by the normalized recovery rate (NRR). Experiments on GridWorld environments ranging from 7 × 7 to 13 × 13 show that, at a privacy level comparable to a strong fixed budget, the proposed dynamic budget raises the reward retention from 6.2% to 78.9% while keeping a comparable NRR reduction. These results indicate that adapting the privacy budget to per-state policy determinism yields a markedly better privacy–utility trade-off than existing training-perturbation-based methods. We clarify that the differential privacy property invoked here is a per-query, mechanism-level guarantee of indistinguishability between neighboring observation states under a fixed trained model. The protection of the environment structure itself is established empirically through the policy inversion attack. Experiments are conducted on discrete GridWorld navigation tasks; the conclusions are scoped to small value-based navigation settings.
Funding Information
  • Guizhou Provincial Science and Technology Department (QianKeHe Basic MS [2025] 622)
  • Guizhou Provincial Science and Technology Department (QianKeHe Basic—[2024] Youth 126)

This publication has 18 references indexed in Scilit: