Emergence of cooperation under punishment: A reinforcement learning perspective
- 1 July 2024
- journal article
- research article
- Published by AIP Publishing in Chaos: An Interdisciplinary Journal of Nonlinear Science
- Vol. 34 (7)
Abstract
Punishment is a common tactic to sustain cooperation and has been extensively studied for a long time. While most of previous game-theoretic work adopt the imitation learning framework where players imitate the strategies of those who are better off, the learning logic in the real world is often much more complex. In this work, we turn to the reinforcement learning paradigm, where individuals make their decisions based upon their experience and long-term returns. Specifically, we investigate the prisoners’ dilemma game with a Q-learning algorithm, and cooperators probabilistically pose punishment on defectors in their neighborhood. Unexpectedly, we find that punishment could lead to either continuous or discontinuous cooperation phase transitions, and the nucleation process of cooperation clusters is reminiscent of the liquid–gas transition. The analysis of a Q-table reveals the evolution of the underlying “psychologic” changes, which explains the nucleation process and different levels of cooperation. The uncovered first-order phase transition indicates that great care needs to be taken when implementing the punishment compared to the continuous scenario.Keywords
All Related Versions
Funding Information
- National Natural Science Foundation of China (12075144)
- National Natural Science Foundation of China (12165014)
- Fundamental Research Funds for the Central Universities (GK202401002)
This publication has 65 references indexed in Scilit:
- Emergence of cooperation in two-agent repeated games with reinforcement learningChaos, Solitons & Fractals, 2023
- Impact of peer pressure on the evolution of cooperation in prisoner’s dilemma gameChaos, Solitons & Fractals, 2023
- Emergence of cooperation in a population with bimodal response behaviorsChaos, Solitons & Fractals, 2023
- Lévy noise promotes cooperation in the prisoner’s dilemma game with reinforcement learningNonlinear Dynamics, 2022
- Particle swarm intelligence and the evolution of cooperation in the spatial public goods game with punishmentApplied Mathematics and Computation, 2022
- Effectiveness of conditional punishment for the evolution of public cooperationJournal of Theoretical Biology, 2013
- Repeated games and direct reciprocity under active linkingJournal of Theoretical Biology, 2008
- The leading eight: Social norms that can maintain cooperation by indirect reciprocityJournal of Theoretical Biology, 2006
- The Dynamics of Indirect ReciprocityJournal of Theoretical Biology, 1998
- Q-learningMachine Learning, 1992