Evolution of fairness in multi-objective reinforcement learning framework
Authors:
- Jingyi ZhangJZJingyi Zhang
- Xin OuXOXin Ou
- Guo-Zhong ZhengGZGuo-Zhong Zheng
- Shengfeng DengSDShengfeng Deng
- Ji-Qiang ZhangJZJi-Qiang Zhang
- Li ChenLCLi Chen
Preprint
- 18 October 2026
- preprint
- Published by Elsevier in SSRN Electronic Journal
Abstract
Fairness, as a fundamental social norm, continues to pose a longstanding puzzle regarding its emergence. Traditional game-theoretic models largely rely on the assumption of Homo economicus, wherein individuals are purely rational and self-interested, acting solely to maximize material payoffs. Such accounts, however, overlook the multidimensional nature of human decision-making, which is often shaped also by other considerations beyond economic incentives. To address this gap, we propose a multi-objective reinforcement learning framework that models the evolution of fairness as a dynamic trade-off between material payoff maximization and fairness-driven moral behavior, regulated by a fairness pressure coefficient. Using simulations of a two-objective Q-learning ultimatum game, we find that increased fairness pressure promotes fair outcomes, as expected. Strikingly, however, under moderate pressure, responders become ”forgiving'' by accepting low offers -- a pattern in line with our daily experience. Microscopic analyses reveal that this strategy reversal stems from competition between payoff-maximizing and fairness-oriented preferences. We further extend our framework to an asymmetric setting, where proposers and responders assign different weights to the two objectives. Overall, our work expands the reinforcement learning paradigm from a single-objective to a multi-objective formulation, offering a versatile tool for elucidating a broader range of human social behaviors.Keywords
This publication has 45 references indexed in Scilit:
- Evolution of cooperation in the public goods game with Q-learningChaos, Solitons & Fractals, 2024
- Self-serving bias in fairness perception: Allowing allocators to allocate unfairlyCurrent Psychology, 2023
- Reputation update of responders efficiently promotes the evolution of fairness in the ultimatum gameChaos, Solitons & Fractals, 2023
- Modeling collective motion for fish schooling via multi-agent reinforcement learningEcological Modelling, 2022
- A practical guide to multi-objective reinforcement learning and planningAutonomous Agents and Multi-Agent Systems, 2022
- The dynamics of responder behavior in ultimatum games: a meta-studyExperimental Economics, 2011
- Multi-objective Optimisation Using Evolutionary Algorithms: An Introduction2011
- On scalarizing functions in multiobjective optimizationOR Spectrum, 2002
- Anonymity versus Punishment in Ultimatum BargainingGames and Economic Behavior, 1995
- An experimental analysis of ultimatum bargainingJournal of Economic Behavior & Organization, 1982