The emergence of cooperation via Q-learning in spatial donation game
Open Access
- 26 April 2024
- journal article
- research article
- Published by IOP Publishing in Journal of Physics: Complexity
- Vol. 5 (2) , 025006
Abstract
Decision-making often overlooks the feedback between agents and the environment. Reinforcement learning is widely employed through exploratory experimentation to address problems related to states, actions, rewards, decision-making in various contexts. This work considers a new perspective, where individuals continually update their policies based on interactions with the spatial environment, aiming to maximize cumulative rewards and learn the optimal strategy. Specifically, we utilize the Q-learning algorithm to study the emergence of cooperation in a spatial population playing the donation game. Each individual has a Q-table that guides their decision-making in the game. Interestingly, we find that cooperation emerges within this introspective learning framework, and a smaller learning rate and higher discount factor make cooperation more likely to occur. Through the analysis of Q-table evolution, we disclose the underlying mechanism for cooperation, which may provide some insights to the emergence of cooperation in the real-world systems.Keywords
Funding Information
- National Natural Science Foundation of China (No.12075144)
- Project of Science and Technology Commission of Shanghai Municipality (No. 22JC1401403)
This publication has 64 references indexed in Scilit:
- Interaction state Q-learning promotes cooperation in the spatial prisoner's dilemma gameApplied Mathematics and Computation, 2023
- Does Spending More Always Ensure Higher Cooperation? An Analysis of Institutional Incentives on Heterogeneous NetworksDynamic Games and Applications, 2023
- Lévy noise promotes cooperation in the prisoner’s dilemma game with reinforcement learningNonlinear Dynamics, 2022
- The rise and fall of donation behavior through reputationChaos, Solitons & Fractals, 2021
- Symmetric equilibrium of multi-agent reinforcement learning in repeated prisoner’s dilemmaApplied Mathematics and Computation, 2021
- Learning continuous and consistent strategy promotes cooperation in prisoner’s dilemma game with mixed strategyApplied Mathematics and Computation, 2020
- Oscillatory evolution of collective behavior in evolutionary games played with reinforcement learningNonlinear Dynamics, 2020
- Donating to disaster victims: Responses to natural and humanly caused eventsEuropean Journal of Social Psychology, 2010
- Q-learningMachine Learning, 1992
- The genetical evolution of social behaviour. IIJournal of Theoretical Biology, 1964