RWRichard Willis

Richard Willis

*This is an algorithmically generated author record based on publicly available information.

Current Organizations / Institutes

The organizations/institutes from the scholar's latest article


Richard Willis has a total of 6 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • Open Access

    Evaluating Collective Behaviour of Hundreds of LLM Agents

    by Richard Willis, Jianing Zhao, Yali Du, Joel Z. Leibo

    As autonomous agents powered by LLM are increasingly deployed in society, understanding their collective behaviour in social dilemmas becomes critical. We introduce an evaluation framework where LLMs generate strategies encoded as algorithms, enabling inspection prior to deployment and scaling to populations of hundreds of agents -- substantially larger than in previous work. We find that more recent models tend to produce worse societal outcomes compared to older models when agents prioritise individual gain over collective benefits. Using cultural evolution to model user selection of agents, our simulations reveal a significant risk of convergence to poor societal equilibria, particularly when the relative benefit of cooperation diminishes and population sizes increase. We release our code as an evaluation suite for developers to assess the emergent collective behaviour of their models.

  • Open Access

    Quantifying the Self-Interest Level of Markov Social Dilemmas

    by Richard Willis, Yali Du, Joel Z. Leibo, Michael Luck

    This paper introduces a novel method for estimating the self-interest level of Markov social dilemmas. We extend the concept of self-interest level from normal-form games to Markov games, providing a quantitative measure of the minimum reward exchange required to align individual and collective interests. We demonstrate our method on three environments from the Melting Pot suite, representing either common-pool resources or public goods. Our results illustrate how reward exchange can enable agents to transition from selfish to collective equilibria in a Markov social dilemma. This work contributes to multi-agent reinforcement learning by providing a practical tool for analysing complex, multistep social dilemmas. Our findings offer insights into how reward structures can promote or hinder cooperation, with potential applications in areas such as mechanism design.


  • Open Access

    Will Systems of LLM Agents Cooperate: An Investigation into a Social Dilemma

    by Richard Willis, Yali Du, Joel Z Leibo, Michael Luck

    As autonomous agents become more prevalent, understanding their collective behaviour in strategic interactions is crucial. This study investigates the emergent cooperative tendencies of systems of Large Language Model (LLM) agents in a social dilemma. Unlike previous research where LLMs output individual actions, we prompt state-of-the-art LLMs to generate complete strategies for iterated Prisoner's Dilemma. Using evolutionary game theory, we simulate populations of agents with different strategic dispositions (aggressive, cooperative, or neutral) and observe their evolutionary dynamics. Our findings reveal that different LLMs exhibit distinct biases affecting the relative success of aggressive versus cooperative strategies. This research provides insights into the potential long-term behaviour of systems of deployed LLM-based autonomous agents and highlights the importance of carefully considering the strategic environments in which they operate.


  • Open Access

    Resolving social dilemmas with minimal reward transfer

    by Richard Willis, Yali Du, Joel Z. Leibo, Michael Luck

    Social dilemmas present a significant challenge in multi-agent cooperation because individuals are incentivised to behave in ways that undermine socially optimal outcomes. Consequently, self-interested agents often avoid collective behaviour. In response, we formalise social dilemmas and introduce a novel metric, the general self-interest level, to quantify the disparity between individual and group rationality in such scenarios. This metric represents the maximum proportion of their individual rewards that agents can retain while ensuring that a social welfare optimum becomes a dominant strategy. Our approach diverges from traditional concepts of altruism, instead focusing on strategic reward redistribution. By transferring rewards among agents in a manner that aligns individual and group incentives, rational agents will maximise collective welfare while pursuing their own interests. We provide an algorithm to compute efficient transfer structures for an arbitrary number of agents, and introduce novel multi-player social dilemma games to illustrate the effectiveness of our method. This work provides both a descriptive tool for analysing social dilemmas and a prescriptive solution for resolving them via efficient reward transfer contracts. Applications include mechanism design, where we can assess the impact on collaborative behaviour of modifications to models of environments.


  • Open Access

    A Review of Cooperation in Multi-agent Learning

    by Yali Du, Joel Z. Leibo, Usman Islam, Richard Willis, Peter Sunehag

    Cooperation in multi-agent learning (MAL) is a topic at the intersection of numerous disciplines, including game theory, economics, social sciences, and evolutionary biology. Research in this area aims to understand both how agents can coordinate effectively when goals are aligned and how they may cooperate in settings where gains from working together are possible but possibilities for conflict abound. In this paper we provide an overview of the fundamental concepts, problem settings and algorithms of multi-agent learning. This encompasses reinforcement learning, multi-agent sequential decision-making, challenges associated with multi-agent cooperation, and a comprehensive review of recent progress, along with an evaluation of relevant metrics. Finally we discuss open challenges in the field with the aim of inspiring new avenues for research.