Abstract
In many scenarios, a principal dynamically interacts with an agent and offers a sequence of incentives to align the agent's behavior with a desired objective. This paper focuses on the problem of synthesizing an incentive sequence that, once offered, induces the desired agent behavior even when the agent's intrinsic motivation is unknown to the principal. We model the agent's behavior as a Markov decision process, express its intrinsic motivation as a reward function, which belongs to a finite set of possible reward functions, and consider the incentives as additional rewards offered to the agent. We first show that the behavior modification problem (BMP), i.e., the problem of synthesizing an incentive sequence that induces a desired agent behavior at minimum total cost to the principal, is PSPACE-hard. Moreover, we obtain two NP-complete variants of the BMP by imposing certain restrictions on the incentive sequences available to the principal. We also provide a sufficient condition on the set of possible reward functions under which the BMP can be solved via linear programming. Finally, we propose two algorithms to solve the NP-complete variants of the BMP.
Funding Information
  • Defense Advanced Research Projects Agency (D19AP00004)
  • Air Force Research Laboratory (FA9550-19-1-0169)
  • ARL (W911NF-17-2-0181)
  • ARO (W911NF1910483)
  • Defense Advanced Research Projects Agency (FA8750-20-2-0502)

This publication has 23 references indexed in Scilit: