Ergodic Control of a Heterogeneous Population and Application to Electricity Pricing
- 20 January 2025
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Automatic Control
- Vol. 70 (7) , 4640-4654
Abstract
We consider a control problem for a heterogeneous population composed of agents able to switch at any time between different options. The controller aims to maximize an average gain per time unit, supposing that the population is of infinite size. This leads to an ergodic control problem for a “mean-field” Markov Decision Process in which the state space is a product of simplices, and the population evolves according to controlled linear dynamics. By exploiting contraction properties of the dynamics in Hilbert's projective metric, we prove that the infinite-dimensional ergodic eigenproblem admits a solution and show that the latter is in general non unique. This allows us to obtain optimal strategies, and to quantify the gap between steady-state strategies and optimal ones. In particular, we prove in the one-dimensional case that there exist cyclic policies – alternating between discount and profit taking stages – which secure a greater gain than constant-price policies. On numerical aspects, we develop a policy iteration algorithm with “on-the-fly” generated transitions, specifically adapted to decomposable models, leading to substantial memory savings. We finally apply our results on realistic instances coming from an electricity pricing problem encountered in the retail markets, and numerically observe the emergence of cyclic promotions for sufficient inertia in the customer behavior.Keywords
This publication has 52 references indexed in Scilit:
- Dobrushin’s Ergodicity Coefficient for Markov Operators on ConesIntegral Equations and Operator Theory, 2014
- Mean Field Games and Mean Field Type Control Theory2013
- Inequalities: Theory of Majorization and Its Applications2011
- Markov Decision Processes with Applications to Finance2011
- Remarks on strongly convex functionsAequationes mathematicae, 2010
- Wiley Series in Probability and Statistics2008
- Risk sensitive control of finite state Markov chains in discrete time, with applications to portfolio managementMathematical Methods of Operations Research, 1999
- Idempotent Analysis and Its Applications1997
- On the burnside problem for semigroups of matrices in the (max,+) algebraSemigroup Forum, 1996
- Discrete-Time Markov Control Processes1996