JZJi-Qiang Zhang

Ji-Qiang Zhang

*This is an algorithmically generated author record based on publicly available information.

Current Organizations / Institutes

The organizations/institutes from the scholar's latest article

Previous Organizations / Institutes

The organizations / institutes from the scholar's other articles in the past ten years


Ji-Qiang Zhang has a total of 49 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • Open Access

    Evolution of fairness in multi-objective reinforcement learning framework

    by Jingyi Zhang, Xin Ou, Guo-Zhong Zheng, Shengfeng Deng, Ji-Qiang Zhang, Li Chen

    Fairness, as a fundamental social norm, continues to pose a longstanding puzzle regarding its emergence. Traditional game-theoretic models largely rely on the assumption of Homo economicus, wherein individuals are purely rational and self-interested, acting solely to maximize material payoffs. Such accounts, however, overlook the multidimensional nature of human decision-making, which is often shaped also by other considerations beyond economic incentives. To address this gap, we propose a multi-objective reinforcement learning framework that models the evolution of fairness as a dynamic trade-off between material payoff maximization and fairness-driven moral behavior, regulated by a fairness pressure coefficient. Using simulations of a two-objective Q-learning ultimatum game, we find that increased fairness pressure promotes fair outcomes, as expected. Strikingly, however, under moderate pressure, responders become ”forgiving'' by accepting low offers -- a pattern in line with our daily experience. Microscopic analyses reveal that this strategy reversal stems from competition between payoff-maximizing and fairness-oriented preferences. We further extend our framework to an asymmetric setting, where proposers and responders assign different weights to the two objectives. Overall, our work expands the reinforcement learning paradigm from a single-objective to a multi-objective formulation, offering a versatile tool for elucidating a broader range of human social behaviors.

  • Decoding species coexistence: A reinforcement learning perspective

    by Kaiwen Jiang, Chenyang Zhao, Shengfeng Deng, Weiran Cai, Jiqiang Zhang, Li Chen

    A central goal in ecology is to understand how biodiversity is maintained. Previous theoretical works have employed the rock-paper-scissors (RPS) game as a toy model, demonstrating that population mobility is crucial in determining the species' coexistence. One key prediction is that biodiversity is jeopardized and eventually lost when mobility exceeds a certain value-a conclusion at odds with empirical observations of highly mobile species coexisting in nature. To address this discrepancy, we introduce a joint reinforcement learning framework to study a spatial RPS model, where individuals' mobility for each species is not fixed but is guided by a common experience pool in the form of a Q-table, and its members jointly revise it via a Q-learning algorithm. Our results show that all three species can coexist stably, with extinction probabilities remaining low across a broad range of baseline migration rates. Mechanistic analysis reveals that individuals develop two behavioral tendencies: survival priority (escaping from predators) and predation priority (remaining near prey). While species coexistence emerges from the balance of the two tendencies, their imbalance jeopardizes biodiversity. Notably, there is a symmetry breaking of action preference in a particular state that is responsible for the divergent species densities. Furthermore, when Q-learning species interact with fixed-mobility counterparts, those with adaptive mobility exhibit a significant evolutionary advantage. Our study suggests that joint reinforcement learning offers a promising perspective for uncovering the mechanisms of biodiversity and designing conservation strategies.


  • Optimal coordination of resource: A solution from reinforcement learning

    by Guozhong Zheng, Weiran Cai, Guanxiao Qi, Jiqiang Zhang, Li Chen

    Efficient allocation is important in nature and human society, where individuals frequently compete for limited resources. The Minority Game (MG) is perhaps the simplest toy model to address this issue. However, most previous solutions assume that the strategies are provided a priori and static, failing to capture their adaptive nature. Here we adopt the reinforcement learning paradigm to the MG, where individuals' decision-making is guided by their experience and the expectation of future rewards. By regulating the balance between exploration and exploitation in individual decision-making, the study reveals diverse collective behaviors. We find that the optimal allocation is reached when individuals appreciate both the experience and the rewards in the future and can balance the exploitation of their experiences with exploration by randomly acting. When the balance of the exploitation and exploration is broken, only partial coordination is observed; in some scenarios, anticoordination may occur, a phenomenon where the coordination is even worse than the scenario where every individual acts purely randomly. Mechanism analysis reveals a symmetry-breaking of action preferences that underlines optimal coordination, where resource utilization reaches its maximum, and the dynamics behind its destabilization. These findings are robust to the population size and the resource capacity. Our work thus provides a different solution to the MG and valuable insights into the resource allocation problems in general.


  • Dual reinforcement learning synergy in resource allocation: Emergence of momentum strategy

    by Zhen-Na Zhang, Guo-Zhong Zheng, Li Chen, Chao-Ran Cai, Sheng-Feng Deng, Bin-Quan Li, Ji-Qiang Zhang


  • Decoding fairness: A reinforcement learning perspective

    by Guozhong Zheng, Jiqiang Zhang, Xin Ou, Shengfeng Deng, Li Chen

    Behavioral experiments on the ultimatum game (UG) reveal that we humans prefer fair acts, which contradicts the prediction made in orthodox economics. Existing explanations, however, are mostly attributed to exogenous factors within the imitation learning framework. Here, we adopt the reinforcement learning paradigm, where individuals make their moves aiming to maximize their accumulated rewards. Specifically, we apply Q learning to UG, where each player is assigned two Q tables to guide decisions for the roles of proposer and responder. In a two-player scenario, fairness emerges prominently when both experiences and future rewards are appreciated. In particular, the probability of successful deals increases with higher offers, which aligns with observations in behavioral experiments. Our mechanism analysis reveals that the system undergoes two phases, eventually stabilizing into fair or rational strategies. These results are robust when the rotating role assignment is replaced by a random or fixed manner, or the scenario is extended to a latticed population. Our findings thus conclude that the endogenous incentive is sufficient to explain the emergence of fairness, exogenous factors are unnecessary.


  • Evolution of cooperation with Q-learning: The impact of information perception

    by Guozhong Zheng, Zhenwei Ding, Jiqiang Zhang, Shengfeng Deng, Weiran Cai, Li Chen

    The inherent complexity of human beings manifests in a remarkable diversity of responses to intricate environments, enabling us to approach problems from varied perspectives. However, in the study of cooperation, existing research within the reinforcement learning framework often assumes that individuals have access to identical information when making decisions, which contrasts with the reality that individuals frequently perceive information differently. In this study, we employ the Q-learning algorithm to explore the impact of information perception on the evolution of cooperation in a two-person prisoner’s dilemma game. We demonstrate that the evolutionary processes differ significantly across three distinct information perception scenarios, highlighting the critical role of information structure in the emergence of cooperation. Notably, the asymmetric information scenario reveals a complex dynamical process, including the emergence, breakdown, and reconstruction of cooperation, mirroring psychological shifts observed in human behavior. Our findings underscore the importance of information structure in fostering cooperation, offering new insights into the establishment of stable cooperative relationships among humans.


  • Cooperation in public goods games: Leveraging other-regarding reinforcement learning on hypergraphs

    by Bo-Ying Li, Zhen-Na Zhang, Guo-Zhong Zheng, Chao-Ran Cai, Ji-Qiang Zhang, Li Chen

    Cooperation is a self-organized collective behavior. It plays a significant role in the evolution of both ecosystems and human society. Reinforcement learning is different from imitation learning, offering a new perspective for exploring cooperation mechanisms. However, most existing studies with the public goods game (PGG) employ a self-regarding setup or are on pairwise interaction networks. Players in the real world, however, optimize their policies based not only on their histories but also on the histories of their coplayers, and the game is played in a group manner. In this work, we investigate the evolution of cooperation in the PGG under the other-regarding reinforcement learning evolutionary game on hypergraph by combining the Q-learning algorithm and evolutionary game framework, where other players' action history is incorporated and the game is played on hypergraphs. Our results show that as the synergy factor r[over ̂] increases, the parameter interval divides into three distinct regions-the absence of cooperation, medium cooperation, and high cooperation-accompanied by two abrupt transitions in the cooperation level near r[over ̂]_{1}^{*} and r[over ̂]_{2}^{*}, respectively. Interestingly, we identify regular and anticoordinated chessboard structures in the spatial pattern that positively contribute to the first cooperation transition but adversely affect the second. Furthermore, we provide a theoretical treatment for the first transition with an approximated r[over ̂]_{1}^{*} and reveal that players with a long-sighted perspective and low exploration rate are more likely to reciprocate kindness with each other, thus facilitating the emergence of cooperation. Our findings contribute to understanding the evolution of human cooperation, where other-regarding information and group interactions are commonplace.


  • Catalytic evolution of cooperation in a population with behavioral bimodality

    by Anhui Sheng, Jing Zhang, Guozhong Zheng, Jiqiang Zhang, Weiran Cai, Li Chen

    The remarkable adaptability of humans in response to complex environments is often demonstrated by the context-dependent adoption of different behavioral modes. However, the existing game-theoretic studies mostly focus on the single-mode assumption, and the impact of this behavioral multimodality on the evolution of cooperation remains largely unknown. Here, we study how cooperation evolves in a population with two behavioral modes. Specifically, we incorporate Q-learning and Tit-for-Tat (TFT) rules into our toy model and investigate the impact of the mode mixture on the evolution of cooperation. While players in a Q-learning mode aim to maximize their accumulated payoffs, players within a TFT mode repeat what their neighbors have done to them. In a structured mixing implementation where the updating rule is fixed for each individual, we find that the mode mixture greatly promotes the overall cooperation prevalence. The promotion is even more significant in the probabilistic mixing, where players randomly select one of the two rules at each step. Finally, this promotion is robust when players adaptively choose the two modes by a real-time comparison. In all three scenarios, players within the Q-learning mode act as catalyzers that turn the TFT players to be more cooperative and as a result drive the whole population to be highly cooperative. The analysis of Q-tables explains the underlying mechanism of cooperation promotion, which captures the “psychological evolution” in the players’ minds. Our study indicates that the variety of behavioral modes is non-negligible and could be crucial to clarify the emergence of cooperation in the real world.


  • Emergence of cooperation under punishment: A reinforcement learning perspective

    by Chenyang Zhao, Guozhong Zheng, Chun Zhang, Jiqiang Zhang, Li Chen

    Punishment is a common tactic to sustain cooperation and has been extensively studied for a long time. While most of previous game-theoretic work adopt the imitation learning framework where players imitate the strategies of those who are better off, the learning logic in the real world is often much more complex. In this work, we turn to the reinforcement learning paradigm, where individuals make their decisions based upon their experience and long-term returns. Specifically, we investigate the prisoners’ dilemma game with a Q-learning algorithm, and cooperators probabilistically pose punishment on defectors in their neighborhood. Unexpectedly, we find that punishment could lead to either continuous or discontinuous cooperation phase transitions, and the nucleation process of cooperation clusters is reminiscent of the liquid–gas transition. The analysis of a Q-table reveals the evolution of the underlying “psychologic” changes, which explains the nucleation process and different levels of cooperation. The uncovered first-order phase transition indicates that great care needs to be taken when implementing the punishment compared to the continuous scenario.