John M. McNamara

People often react to low probability, high damage events in which many die with strong avoidance behaviour. Indeed, analyses of behaviour following the September 11 terror attacks on New York City suggest that this caused a substantial number of additional, ‘indirect’ deaths as many people avoided flying for 12 months afterwards and took to the relatively risky highways of the US instead. We argue that such responses may have arisen as an adaptation to risks that wipe out a significant proportion of all carriers of an allele if they strike, e.g. storms. These are environmental fluctuations known as environmental or aggregate risks. At the opposite extreme, demographic risks affect individuals independently. We show that evolution by natural selection in fluctuating environments means it is adaptive to inflate environmental (aggregate) risks relative to demographic risks, where the inflation factor depends on the proportion of carriers of the allele that die if the risk strikes.

The formation of habits, whereby learnt actions come to be performed automatically with repetition and practice, is a well-established focus for studies in psychology. This contrasts with evolutionarily motivated studies of learning, which typically view behavior as either learnt or fixed, to elucidate the ecological conditions where each predominates. Here, we envisage habit formation functioning to free up limited mental resources (e.g., attention), potentially improving an individual's ability to multitask. As an ecologically relevant case, we investigate exploration-exploitation in foraging under predation risk. In our model, a forager does not know the quality of feeding options, but can learn from the rewards they give. When the environment occasionally changes, individuals can attend to exploration of feeding options in the new conditions. The options can then become habitually exploited, freeing attention for antipredatory vigilance. Via evolutionary simulations, we show that evolutionarily stable forming and breaking of foraging habits can substantially reduce mortality from predation, without drastically reducing foraging success, when environmental conditions remain stable enough between changes. We identify factors promoting habit formation, such as the repetition of actions that yield predictable rewards, and also discuss the role of habits more generally, including their relation to ideas about bounded rationality in theories of decision making. In conclusion, we argue that exploration-exploitation strategies involving the forming and breaking of habits are a type of behavioral flexibility that is likely to be selected for under a range of ecological conditions.

Learning is a taxonomically widespread process by which animals change their behavioural responses to stimuli as a result of experience. In this way, it plays a crucial role in the development of individual behaviour and underpins substantial phenotypic variation within populations. Nevertheless, the impact of learning in social contexts on evolutionary change is not well understood. Here, we develop game theoretical models of competition for resources in small groups (e.g. producer–scrounger and hawk–dove games) in which actions are controlled by reinforcement learning and show that biases in the subjective valuation of different actions readily evolve. Moreover, in many cases, the convergence stable levels of bias exist at fitness minima and therefore lead to disruptive selection on learning rules and, potentially, to the evolution of genetic polymorphisms. Thus, we show how reinforcement learning in social contexts can be a driver of evolutionary diversification. In addition, we consider the evolution of ability in our games, showing that learning can also drive disruptive selection on the ability to perform a task.

Game theory in biology gained prominence 50 years ago, when Maynard Smith & Price formulated the concept of an evolutionarily stable strategy (ESS). Their aim was to explain why conflicts between animals of the same species usually are of a ‘limited war’ type, not causing serious injury. They emphasized that game theory is an alternative to previous ideas about group selection, which were used by ethologists to explain limited aggression. Subsequently, the ESS concept was applied to many phenomena with frequency dependence in the evolutionary success of strategies, including sex allocation, alternative mating types, contest behaviour and signalling, cooperation, and parental care. Both the analyses of signalling and cooperation were inspired by similar problems in economics and attracted much attention in biology. Here we give a perspective on which of the ambitions in the field have been achieved, with a focus on contest behaviour and cooperation. We evaluate whether the game-theoretical study of the evolution of cooperation has measured up to expectations in explaining the behaviour of non-human animals. We also point to potentially fruitful directions for the field, and emphasize the importance of incorporating realistic behavioural mechanisms into models. This article is part of the theme issue ‘Half a century of evolutionary games: a synthesis of theory, application and future directions’.

Animals, including humans, differ in a wide range of physical and cognitive abilities ranging from measures of running speed and physical strength to learning ability and intelligence. We consider the evolution of ability when individuals interact pairwise over their contribution to a common good. In this interaction, the contribution of each is assumed to be the best given their own ability and the contribution of their partner. Since there is a tendency for individuals to partially compensate for a low contribution by their partner, low-ability individuals can do well. As a consequence, for benefit and cost structures for which individuals have a strong response to partner’s contribution, there can be selection for reduced ability. Furthermore, there can be disruptive selection on ability, leading to a bimodal distribution of ability under some modes of inheritance.

Interactions in social groups can promote behavioural specialization. One way this can happen is when individuals engage in activities with two behavioural options and learn which option to choose. We analyse interactions in groups where individuals learn from playing games with two actions and negatively frequency-dependent payoffs, such as producer–scrounger, caller–satellite, or hawk–dove games. Group members are placed in social networks, characterized by the group size and the number of neighbours to interact with, ranging from just a few neighbours to interactions between all group members. The networks we analyse include ring lattices and the much-studied small-world networks. By implementing two basic reinforcement-learning approaches, action–value learning and actor–critic learning, in different games, we find that individuals often show behavioural specialization. Specialization develops more rapidly when there are few neighbours in a network and when learning rates are high. There can be learned specialization also with many neighbours, but we show that, for action–value learning, behavioural consistency over time is higher with a smaller number of neighbours. We conclude that frequency-dependent competition for resources is a main driver of specialization. We discuss our theoretical results in relation to experimental and field observations of behavioural specialization in social situations.

The idea of applying game theory to problems in biology was given a formal basis nearly 50 years ago. Since then, the theory has advanced, and there have been numerous applications of it to...

Most analyses of the origins of cultural evolution focus on when and where social learning prevails over individual learning, overlooking the fact that there are other developmental inputs that influence phenotypic fit to the selective environment. This raises the question of how the presence of other cue ‘channels’ affects the scope for social learning. Here, we present a model that considers the simultaneous evolution of (i) multiple forms of social learning (involving vertical or horizontal learning based on either prestige or conformity biases) within the broader context of other evolving inputs on phenotype determination, including (ii) heritable epigenetic factors, (iii) individual learning, (iv) environmental and cascading maternal effects, (v) conservative bet-hedging, and (vi) genetic cues. In fluctuating environments that are autocorrelated (and hence predictable), we find that social learning from members of the same generation (horizontal social learning) explains the large majority of phenotypic variation, whereas other cues are much less important. Moreover, social learning based on prestige biases typically prevails in positively autocorrelated environments, whereas conformity biases prevail in negatively autocorrelated environments. Only when environments are unpredictable or horizontal social learning is characterized by an intrinsically low information content, other cues such as conservative bet-hedging or vertical prestige biases prevail. This article is part of the theme issue ‘Foundations of cultural evolution’.

We focus on learning during development in a group of individuals that play a competitive game with each other. The game has two actions and there is negative frequency dependence. We define the distribution of actions by group members to be an equilibrium configuration if no individual can improve its payoff by unilaterally changing its action. We show that at this equilibrium, one action is preferred in the sense that those taking the preferred action have a higher payoff than those taking the other, more prosocial, action. We explore the consequences of a simple ‘unbiased’ reinforcement learning rule during development, showing that groups reach an approximate equilibrium distribution, so that some achieve a higher payoff than others. Because there is learning, an individual’s behaviour can influence the future behaviour of others. We show that, as a consequence, there is the potential for an individual to exploit others by influencing them to be the ones to take the non-preferred action. Using an evolutionary simulation, we show that population members can avoid being exploited by over-valuing rewards obtained from the preferred option during learning, an example of a bias that is ‘rational’.