Joel Z. Leibo
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Joel Z. Leibo has a total of 192 co-authors. Here are the top co-authors by number of shared publications:
- Thore Graepel
Guy Lever DeepMind London London DeepMind London DeepMind London DeepMind London UK
6 shared publications Tomaso Poggio
6 shared publications- Edgar A. Duenez-Guzman
Google DeepMind
5 shared publications - Edward Hughes
Google DeepMind, London, UK
5 shared publications - Richard Willis
King's College London
5 shared publications - Yali Du
King's College London The Alan Turing Institute ♣ Shanghai Jiao Tong University
5 shared publications
Latest Publications
- Open Access
Evaluating Collective Behaviour of Hundreds of LLM Agents
As autonomous agents powered by LLM are increasingly deployed in society, understanding their collective behaviour in social dilemmas becomes critical. We introduce an evaluation framework where LLMs generate strategies encoded as algorithms, enabling inspection prior to deployment and scaling to populations of hundreds of agents -- substantially larger than in previous work. We find that more recent models tend to produce worse societal outcomes compared to older models when agents prioritise individual gain over collective benefits. Using cultural evolution to model user selection of agents, our simulations reveal a significant risk of convergence to poor societal equilibria, particularly when the relative benefit of cooperation diminishes and population sizes increase. We release our code as an evaluation suite for developers to assess the emergent collective behaviour of their models.
Open AccessEvaluating Collective Behaviour of Hundreds of LLM Agents
As autonomous agents powered by LLM are increasingly deployed in society, understanding their collective behaviour in social dilemmas becomes critical. We introduce an evaluation framework where LLMs generate strategies encoded as algorithms, enabling inspection prior to deployment and scaling to populations of hundreds of agents -- substantially larger than in previous work. We find that more recent models tend to produce worse societal outcomes compared to older models when agents prioritise individual gain over collective benefits. Using cultural evolution to model user selection of agents, our simulations reveal a significant risk of convergence to poor societal equilibria, particularly when the relative benefit of cooperation diminishes and population sizes increase. We release our code as an evaluation suite for developers to assess the emergent collective behaviour of their models.
- Open Access
A Pragmatic View of AI Personhood
The emergence of agentic Artificial Intelligence (AI) is set to trigger a "Cambrian explosion" of new kinds of personhood. This paper proposes a pragmatic framework for navigating this diversification by treating personhood not as a metaphysical property to be discovered, but as a flexible bundle of obligations (rights and responsibilities) that societies confer upon entities for a variety of reasons, especially to solve concrete governance problems. We argue that this traditional bundle can be unbundled, creating bespoke solutions for different contexts. This will allow for the creation of practical tools -- such as facilitating AI contracting by creating a target "individual" that can be sanctioned -- without needing to resolve intractable debates about an AI's consciousness or rationality. We explore how individuals fit in to social roles and discuss the use of decentralized digital identity technology, examining both "personhood as a problem", where design choices can create "dark patterns" that exploit human social heuristics, and "personhood as a solution", where conferring a bundle of obligations is necessary to ensure accountability or prevent conflict. By rejecting foundationalist quests for a single, essential definition of personhood, this paper offers a more pragmatic and flexible way to think about integrating AI agents into our society.
Open AccessA Pragmatic View of AI Personhood
The emergence of agentic Artificial Intelligence (AI) is set to trigger a "Cambrian explosion" of new kinds of personhood. This paper proposes a pragmatic framework for navigating this diversification by treating personhood not as a metaphysical property to be discovered, but as a flexible bundle of obligations (rights and responsibilities) that societies confer upon entities for a variety of reasons, especially to solve concrete governance problems. We argue that this traditional bundle can be unbundled, creating bespoke solutions for different contexts. This will allow for the creation of practical tools -- such as facilitating AI contracting by creating a target "individual" that can be sanctioned -- without needing to resolve intractable debates about an AI's consciousness or rationality. We explore how individuals fit in to social roles and discuss the use of decentralized digital identity technology, examining both "personhood as a problem", where design choices can create "dark patterns" that exploit human social heuristics, and "personhood as a solution", where conferring a bundle of obligations is necessary to ensure accountability or prevent conflict. By rejecting foundationalist quests for a single, essential definition of personhood, this paper offers a more pragmatic and flexible way to think about integrating AI agents into our society.
- Open Access
- 1 September 2025
- Published by International Joint Conferences on Artificial Intelligence
- Vol. 1, 284-292
Quantifying the Self-Interest Level of Markov Social Dilemmas
This paper introduces a novel method for estimating the self-interest level of Markov social dilemmas. We extend the concept of self-interest level from normal-form games to Markov games, providing a quantitative measure of the minimum reward exchange required to align individual and collective interests. We demonstrate our method on three environments from the Melting Pot suite, representing either common-pool resources or public goods. Our results illustrate how reward exchange can enable agents to transition from selfish to collective equilibria in a Markov social dilemma. This work contributes to multi-agent reinforcement learning by providing a practical tool for analysing complex, multistep social dilemmas. Our findings offer insights into how reward structures can promote or hinder cooperation, with potential applications in areas such as mechanism design.
- 1 September 2025
- Published by International Joint Conferences on Artificial Intelligence
- Vol. 1, 284-292
Open AccessQuantifying the Self-Interest Level of Markov Social Dilemmas
This paper introduces a novel method for estimating the self-interest level of Markov social dilemmas. We extend the concept of self-interest level from normal-form games to Markov games, providing a quantitative measure of the minimum reward exchange required to align individual and collective interests. We demonstrate our method on three environments from the Melting Pot suite, representing either common-pool resources or public goods. Our results illustrate how reward exchange can enable agents to transition from selfish to collective equilibria in a Markov social dilemma. This work contributes to multi-agent reinforcement learning by providing a practical tool for analysing complex, multistep social dilemmas. Our findings offer insights into how reward structures can promote or hinder cooperation, with potential applications in areas such as mechanism design.
- Open Access
- 16 June 2025
- Published by National Academy of Sciences in Proceedings of the National Academy of Sciences
- Vol. 122 (25)
Collective cooperative intelligence
Cooperation at scale is critical for achieving a sustainable future for humanity. However, achieving collective, cooperative behavior—in which intelligent actors in complex environments jointly improve their well-being—remains poorly understood. Complex systems science (CSS) provides a rich understanding of collective phenomena, the evolution of cooperation, and the institutions that can sustain both. Yet, much of the theory in this area fails to fully consider individual-level complexity and environmental context—largely for the sake of tractability and because it has not been clear how to do so rigorously. These elements are well captured in multiagent reinforcement learning (MARL), which has recently put focus on cooperative (artificial) intelligence. However, typical MARL simulations can be computationally expensive and challenging to interpret. In this perspective, we propose that bridging CSS and MARL affords new directions forward. Both fields can complement each other in their goals, methods, and scope. MARL offers CSS concrete ways to formalize cognitive processes in dynamic environments. CSS offers MARL improved qualitative insight into emergent collective phenomena. We see this approach as providing the necessary foundations for a proper science of collective, cooperative intelligence. We highlight work that is already heading in this direction and discuss concrete steps for future research.
- 16 June 2025
- Published by National Academy of Sciences in Proceedings of the National Academy of Sciences
- Vol. 122 (25)
Open AccessCollective cooperative intelligence
Cooperation at scale is critical for achieving a sustainable future for humanity. However, achieving collective, cooperative behavior—in which intelligent actors in complex environments jointly improve their well-being—remains poorly understood. Complex systems science (CSS) provides a rich understanding of collective phenomena, the evolution of cooperation, and the institutions that can sustain both. Yet, much of the theory in this area fails to fully consider individual-level complexity and environmental context—largely for the sake of tractability and because it has not been clear how to do so rigorously. These elements are well captured in multiagent reinforcement learning (MARL), which has recently put focus on cooperative (artificial) intelligence. However, typical MARL simulations can be computationally expensive and challenging to interpret. In this perspective, we propose that bridging CSS and MARL affords new directions forward. Both fields can complement each other in their goals, methods, and scope. MARL offers CSS concrete ways to formalize cognitive processes in dynamic environments. CSS offers MARL improved qualitative insight into emergent collective phenomena. We see this approach as providing the necessary foundations for a proper science of collective, cooperative intelligence. We highlight work that is already heading in this direction and discuss concrete steps for future research.
- Open Access
- 16 June 2025
- Published by National Academy of Sciences in Proceedings of the National Academy of Sciences
- Vol. 122 (25)
Tabula rasa agents display emergent in-group behavior
Theories on group-bias often posit an internal preparedness to bias one’s cognition to favor the in-group (often envisioned as a product of evolution). In contrast, other theories suggest that group-biases can emerge from nonspecialized cognitive processes. These perspectives have historically been difficult to disambiguate given that observed behavior can often be attributed to innate processes, even when groups are experimentally assigned. Here, we use modern techniques from the field of AI that allow us to ask what group biases can be expected from a learning agent that is a pure blank slate without any intrinsic social biases, and whose lifetime of experiences can be tightly controlled. This is possible because deep reinforcement-learning agents learn to convert raw sensory input (i.e. pixels) to reward-driven action, a unique feature among cognitive models. We find that blank slate agents do develop group biases based on arbitrary group differences (i.e. color). We show that the bias develops as a result of familiarity of experience and depends on the visual patterns becoming associated with reward through interaction. The bias artificial agents display is not a static reflection of the bias in their stream of experiences. In this minimal environment, the bias can be overcome given enough positive experiences, although unlearning the bias takes longer than acquiring it. Further, we show how this style of tabula rasa group behavior model can be used to test fine-grained predictions of psychological theories.
- 16 June 2025
- Published by National Academy of Sciences in Proceedings of the National Academy of Sciences
- Vol. 122 (25)
Open AccessTabula rasa agents display emergent in-group behavior
Theories on group-bias often posit an internal preparedness to bias one’s cognition to favor the in-group (often envisioned as a product of evolution). In contrast, other theories suggest that group-biases can emerge from nonspecialized cognitive processes. These perspectives have historically been difficult to disambiguate given that observed behavior can often be attributed to innate processes, even when groups are experimentally assigned. Here, we use modern techniques from the field of AI that allow us to ask what group biases can be expected from a learning agent that is a pure blank slate without any intrinsic social biases, and whose lifetime of experiences can be tightly controlled. This is possible because deep reinforcement-learning agents learn to convert raw sensory input (i.e. pixels) to reward-driven action, a unique feature among cognitive models. We find that blank slate agents do develop group biases based on arbitrary group differences (i.e. color). We show that the bias develops as a result of familiarity of experience and depends on the visual patterns becoming associated with reward through interaction. The bias artificial agents display is not a static reflection of the bias in their stream of experiences. In this minimal environment, the bias can be overcome given enough positive experiences, although unlearning the bias takes longer than acquiring it. Further, we show how this style of tabula rasa group behavior model can be used to test fine-grained predictions of psychological theories.
- Open Access
- 16 June 2025
- Published by National Academy of Sciences in Proceedings of the National Academy of Sciences
- Vol. 122 (25)
Perceptual interventions ameliorate statistical discrimination in learning agents
Choosing social partners is a potentially demanding task which involves paying attention to the right information while disregarding salient but possibly irrelevant features. The resultant trade-off between cost of evaluation and quality of decisions can lead to undesired bias. Information-processing abilities mediate this trade-off, where individuals with higher ability choose better partners leading to higher performance. By altering the salience of features, technology can modulate the effect of information-processing limits, potentially increasing or decreasing undesired biases. Here, we use game theory and multiagent reinforcement learning to investigate how undesired biases emerge, and how a technological layer (in the form of a perceptual intervention) between individuals and their environment can ameliorate such biases. Our results show that a perceptual intervention designed to increase the salience of outcome-relevant features can reduce bias in agents making partner choice decisions. Individuals learning with a perceptual intervention showed less bias due to decreased reliance on features that only spuriously correlate with behavior. Mechanistically, the perceptual intervention effectively increased the information-processing abilities of the individuals. Our results highlight the benefit of using multiagent reinforcement learning to model theoretically grounded social behaviors, particularly when real-world complexity prohibits fully analytical approaches.
- 16 June 2025
- Published by National Academy of Sciences in Proceedings of the National Academy of Sciences
- Vol. 122 (25)
Open AccessPerceptual interventions ameliorate statistical discrimination in learning agents
Choosing social partners is a potentially demanding task which involves paying attention to the right information while disregarding salient but possibly irrelevant features. The resultant trade-off between cost of evaluation and quality of decisions can lead to undesired bias. Information-processing abilities mediate this trade-off, where individuals with higher ability choose better partners leading to higher performance. By altering the salience of features, technology can modulate the effect of information-processing limits, potentially increasing or decreasing undesired biases. Here, we use game theory and multiagent reinforcement learning to investigate how undesired biases emerge, and how a technological layer (in the form of a perceptual intervention) between individuals and their environment can ameliorate such biases. Our results show that a perceptual intervention designed to increase the salience of outcome-relevant features can reduce bias in agents making partner choice decisions. Individuals learning with a perceptual intervention showed less bias due to decreased reliance on features that only spuriously correlate with behavior. Mechanistically, the perceptual intervention effectively increased the information-processing abilities of the individuals. Our results highlight the benefit of using multiagent reinforcement learning to model theoretically grounded social behaviors, particularly when real-world complexity prohibits fully analytical approaches.
- Open Access
Societal and technological progress as sewing an ever-growing, ever-changing, patchy, and polychrome quilt
Artificial Intelligence (AI) systems are increasingly placed in positions where their decisions have real consequences, e.g., moderating online spaces, conducting research, and advising on policy. Ensuring they operate in a safe and ethically acceptable fashion is thus critical. However, most solutions have been a form of one-size-fits-all "alignment". We are worried that such systems, which overlook enduring moral diversity, will spark resistance, erode trust, and destabilize our institutions. This paper traces the underlying problem to an often-unstated Axiom of Rational Convergence: the idea that under ideal conditions, rational agents will converge in the limit of conversation on a single ethics. Treating that premise as both optional and doubtful, we propose what we call the appropriateness framework: an alternative approach grounded in conflict theory, cultural evolution, multi-agent systems, and institutional economics. The appropriateness framework treats persistent disagreement as the normal case and designs for it by applying four principles: (1) contextual grounding, (2) community customization, (3) continual adaptation, and (4) polycentric governance. We argue here that adopting these design principles is a good way to shift the main alignment metaphor from moral unification to a more productive metaphor of conflict management, and that taking this step is both desirable and urgent.
Open AccessSocietal and technological progress as sewing an ever-growing, ever-changing, patchy, and polychrome quilt
Artificial Intelligence (AI) systems are increasingly placed in positions where their decisions have real consequences, e.g., moderating online spaces, conducting research, and advising on policy. Ensuring they operate in a safe and ethically acceptable fashion is thus critical. However, most solutions have been a form of one-size-fits-all "alignment". We are worried that such systems, which overlook enduring moral diversity, will spark resistance, erode trust, and destabilize our institutions. This paper traces the underlying problem to an often-unstated Axiom of Rational Convergence: the idea that under ideal conditions, rational agents will converge in the limit of conversation on a single ethics. Treating that premise as both optional and doubtful, we propose what we call the appropriateness framework: an alternative approach grounded in conflict theory, cultural evolution, multi-agent systems, and institutional economics. The appropriateness framework treats persistent disagreement as the normal case and designs for it by applying four principles: (1) contextual grounding, (2) community customization, (3) continual adaptation, and (4) polycentric governance. We argue here that adopting these design principles is a good way to shift the main alignment metaphor from moral unification to a more productive metaphor of conflict management, and that taking this step is both desirable and urgent.
- Open Access
Will Systems of LLM Agents Cooperate: An Investigation into a Social Dilemma
As autonomous agents become more prevalent, understanding their collective behaviour in strategic interactions is crucial. This study investigates the emergent cooperative tendencies of systems of Large Language Model (LLM) agents in a social dilemma. Unlike previous research where LLMs output individual actions, we prompt state-of-the-art LLMs to generate complete strategies for iterated Prisoner's Dilemma. Using evolutionary game theory, we simulate populations of agents with different strategic dispositions (aggressive, cooperative, or neutral) and observe their evolutionary dynamics. Our findings reveal that different LLMs exhibit distinct biases affecting the relative success of aggressive versus cooperative strategies. This research provides insights into the potential long-term behaviour of systems of deployed LLM-based autonomous agents and highlights the importance of carefully considering the strategic environments in which they operate.
Open AccessWill Systems of LLM Agents Cooperate: An Investigation into a Social Dilemma
As autonomous agents become more prevalent, understanding their collective behaviour in strategic interactions is crucial. This study investigates the emergent cooperative tendencies of systems of Large Language Model (LLM) agents in a social dilemma. Unlike previous research where LLMs output individual actions, we prompt state-of-the-art LLMs to generate complete strategies for iterated Prisoner's Dilemma. Using evolutionary game theory, we simulate populations of agents with different strategic dispositions (aggressive, cooperative, or neutral) and observe their evolutionary dynamics. Our findings reveal that different LLMs exhibit distinct biases affecting the relative success of aggressive versus cooperative strategies. This research provides insights into the potential long-term behaviour of systems of deployed LLM-based autonomous agents and highlights the importance of carefully considering the strategic environments in which they operate.
Beyond the matrix: Experimental approaches to studying cognitive agents in social-ecological systems
- Open Access
- 12 October 2024
- Published by Springer Nature in Autonomous Agents and Multi-Agent Systems
- Vol. 38 (2) , 1-33
Resolving social dilemmas with minimal reward transfer
Social dilemmas present a significant challenge in multi-agent cooperation because individuals are incentivised to behave in ways that undermine socially optimal outcomes. Consequently, self-interested agents often avoid collective behaviour. In response, we formalise social dilemmas and introduce a novel metric, the general self-interest level, to quantify the disparity between individual and group rationality in such scenarios. This metric represents the maximum proportion of their individual rewards that agents can retain while ensuring that a social welfare optimum becomes a dominant strategy. Our approach diverges from traditional concepts of altruism, instead focusing on strategic reward redistribution. By transferring rewards among agents in a manner that aligns individual and group incentives, rational agents will maximise collective welfare while pursuing their own interests. We provide an algorithm to compute efficient transfer structures for an arbitrary number of agents, and introduce novel multi-player social dilemma games to illustrate the effectiveness of our method. This work provides both a descriptive tool for analysing social dilemmas and a prescriptive solution for resolving them via efficient reward transfer contracts. Applications include mechanism design, where we can assess the impact on collaborative behaviour of modifications to models of environments.
- 12 October 2024
- Published by Springer Nature in Autonomous Agents and Multi-Agent Systems
- Vol. 38 (2) , 1-33
Open AccessResolving social dilemmas with minimal reward transfer
Social dilemmas present a significant challenge in multi-agent cooperation because individuals are incentivised to behave in ways that undermine socially optimal outcomes. Consequently, self-interested agents often avoid collective behaviour. In response, we formalise social dilemmas and introduce a novel metric, the general self-interest level, to quantify the disparity between individual and group rationality in such scenarios. This metric represents the maximum proportion of their individual rewards that agents can retain while ensuring that a social welfare optimum becomes a dominant strategy. Our approach diverges from traditional concepts of altruism, instead focusing on strategic reward redistribution. By transferring rewards among agents in a manner that aligns individual and group incentives, rational agents will maximise collective welfare while pursuing their own interests. We provide an algorithm to compute efficient transfer structures for an arbitrary number of agents, and introduce novel multi-player social dilemma games to illustrate the effectiveness of our method. This work provides both a descriptive tool for analysing social dilemmas and a prescriptive solution for resolving them via efficient reward transfer contracts. Applications include mechanism design, where we can assess the impact on collaborative behaviour of modifications to models of environments.