Convergence of Heterogeneous Learning Dynamics in Zero-Sum Stochastic Games
- 2 June 2025
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Automatic Control
- Vol. 70 (11) , 7523-7537
Abstract
In this article, we present new families of algorithms for the repeated play of two-agent (near) zero-sum games and two-agent zero-sum stochastic games. For example, the family includes fictitious play and its variants as members. Commonly, the algorithms in this family are all uncoupled, rational, and convergent even in heterogeneous cases, e.g., where the dynamics may differ in terms of learning rates, full, none or temporal access to opponent actions, and model-based versus model-free learning. The convergence of heterogeneous dynamics is of practical interest especially in competitive environments since agents may have no means or interests in following the same dynamic with the same parameters. We prove that any mixture of such asymmetries does not impact the algorithms’ convergence to equilibrium (or near equilibrium if there is experimentation) in zero-sum games with repeated play and in zero-sum (irreducible) stochastic games with sufficiently small discount factors.Keywords
All Related Versions
Funding Information
- The Scientific and Technological Research Council of Türkiye (2232-B)
- International Fellowship for Early Stage Researchers (121C124)
This publication has 39 references indexed in Scilit:
- Stochastic Networked Control Systems2013
- Multi-agent Reinforcement Learning: An Overview2010
- Stochastic Approximation2008
- Experience-Weighted Attraction Learning in Coordination Games: Probability Rules, Heterogeneity, and Time-VariationJournal of Mathematical Psychology, 1998
- On the Rate of Convergence of Continuous-Time Fictitious PlayGames and Economic Behavior, 1998
- Individual Learning in Normal Form Games: Some Laboratory ResultsGames and Economic Behavior, 1997
- Consistency and cautious fictitious playJournal of Economic Dynamics and Control, 1995
- Asynchronous stochastic approximation and Q-learningMachine Learning, 1994
- Markov games as a framework for multi-agent reinforcement learning1994
- Learning Mixed EquilibriaGames and Economic Behavior, 1993