On the Complexity of Sequential Incentive Design
- 2 November 2021
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Automatic Control
- Vol. 67 (11) , 5809-5824
Abstract
In many scenarios, a principal dynamically interacts with an agent and offers a sequence of incentives to align the agent's behavior with a desired objective. This paper focuses on the problem of synthesizing an incentive sequence that, once offered, induces the desired agent behavior even when the agent's intrinsic motivation is unknown to the principal. We model the agent's behavior as a Markov decision process, express its intrinsic motivation as a reward function, which belongs to a finite set of possible reward functions, and consider the incentives as additional rewards offered to the agent. We first show that the behavior modification problem (BMP), i.e., the problem of synthesizing an incentive sequence that induces a desired agent behavior at minimum total cost to the principal, is PSPACE-hard. Moreover, we obtain two NP-complete variants of the BMP by imposing certain restrictions on the incentive sequences available to the principal. We also provide a sufficient condition on the set of possible reward functions under which the BMP can be solved via linear programming. Finally, we propose two algorithms to solve the NP-complete variants of the BMP.Keywords
All Related Versions
Funding Information
- Defense Advanced Research Projects Agency (D19AP00004)
- Air Force Research Laboratory (FA9550-19-1-0169)
- ARL (W911NF-17-2-0181)
- ARO (W911NF1910483)
- Defense Advanced Research Projects Agency (FA8750-20-2-0502)
This publication has 23 references indexed in Scilit:
- Multi-objective Model Checking of Markov Decision Processes2007
- Robust Mechanism DesignEconometrica, 2005
- The Concave-Convex ProcedureNeural Computation, 2003
- Algorithmic Mechanism DesignGames and Economic Behavior, 2001
- Introduction to the Theory of ComputationACM SIGACT News, 1996
- Affine Incentive Schemes for Stochastic Systems with Dynamic InformationSIAM Journal on Control and Optimization, 1984
- A control-theoretic view on incentivesAutomatica, 1982
- The Euclidean travelling salesman problem is NP-completeTheoretical Computer Science, 1977
- Computability of global solutions to factorable nonconvex programs: Part I — Convex underestimating problemsMathematical Programming, 1976
- Dynamic Models for Sales Promotion PoliciesJournal of the Operational Research Society, 1973