Ole Peters
Research Subjects
Ole Peters has a total of 40 co-authors. Here are the top co-authors by number of shared publications:
- Alexander Adamou
Laboratory London Mathematical 8 Margravine Gardens W6 8RH London UK
7 shared publications - Kim Christensen
Imperial College London
4 shared publications - Alexander T. I. Adamou
Department of Mathematics, Imperial College London, South Kensington Campus, London SW7 2AZ, UK
3 shared publications J. David Neelin
Deparment for Atmospheric and Oceanic Sciences, University of California Los Angeles, 405 Hilgard Avenue, LA, CA90095-1565, USA
3 shared publications- Christopher E. Holloway
Department of Meteorology University of Reading Reading UK
2 shared publications Gunnar Pruessner
2 shared publications
Latest Publications
- Open Access
Non-ergodicity in reinforcement learning: robustness via ergodicity transformations
Envisioned application areas for reinforcement learning (RL) include autonomous driving, precision agriculture, and finance, which all require RL agents to make decisions in the real world. A significant challenge hindering the adoption of RL methods in these domains is the non-robustness of conventional algorithms. In this paper, we argue that a fundamental issue contributing to this lack of robustness lies in the focus on the expected value of the return as the sole ``correct'' optimization objective. The expected value is the average over the statistical ensemble of infinitely many trajectories. For non-ergodic returns, this average differs from the average over a single but infinitely long trajectory. Consequently, optimizing the expected value can lead to policies that yield exceptionally high returns with probability zero but almost surely result in catastrophic outcomes. This problem can be circumvented by transforming the time series of collected returns into one with ergodic increments. This transformation enables learning robust policies by optimizing the long-term return for individual agents rather than the average across infinitely many trajectories. We propose an algorithm for learning ergodicity transformations from data and demonstrate its effectiveness in an instructive, non-ergodic environment and on standard RL benchmarks.
Open AccessNon-ergodicity in reinforcement learning: robustness via ergodicity transformations
Envisioned application areas for reinforcement learning (RL) include autonomous driving, precision agriculture, and finance, which all require RL agents to make decisions in the real world. A significant challenge hindering the adoption of RL methods in these domains is the non-robustness of conventional algorithms. In this paper, we argue that a fundamental issue contributing to this lack of robustness lies in the focus on the expected value of the return as the sole ``correct'' optimization objective. The expected value is the average over the statistical ensemble of infinitely many trajectories. For non-ergodic returns, this average differs from the average over a single but infinitely long trajectory. Consequently, optimizing the expected value can lead to policies that yield exceptionally high returns with probability zero but almost surely result in catastrophic outcomes. This problem can be circumvented by transforming the time series of collected returns into one with ergodic increments. This transformation enables learning robust policies by optimizing the long-term return for individual agents rather than the average across infinitely many trajectories. We propose an algorithm for learning ergodicity transformations from data and demonstrate its effectiveness in an instructive, non-ergodic environment and on standard RL benchmarks.
- Open Access
- 3 July 2023
- Published by Cambridge University Press (CUP) in Annals of Actuarial Science
- Vol. 17 (2) , 215-218
Insurance as an ergodicity problem
- 3 July 2023
- Published by Cambridge University Press (CUP) in Annals of Actuarial Science
- Vol. 17 (2) , 215-218
Open AccessInsurance as an ergodicity problem
- Open Access
- 23 May 2022
- Published by The Royal Society in Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences
- Vol. 380 (2227) , 20200425
The ergodicity solution of the cooperation puzzle
When two entities cooperate by sharing resources, one relinquishes something of value to the other. This apparent altruism is frequently observed in nature. Why? Classical treatments assume circumstances where combining resources creates an immediate benefit, e.g. through complementarity or thresholds. Here we ask whether cooperation is predictable without such circumstances. We study a model in which resources self-multiply with fluctuations, a null model of a range of phenomena from viral spread to financial investment. Two fundamental growth rates exist: the ensemble-average growth rate, achieved by the average resources of a large population; and the time-average growth rate, achieved by individual resources over a long time. As a consequence of non-ergodicity, the latter is lower than the former by a term which depends on fluctuation size. Repeated pooling and sharing of resources reduces the effective size of fluctuations and increases the time-average growth rate, which approaches the ensemble-average growth rate in the many-cooperator limit. Therefore, cooperation is advantageous in our model for the simple reason that those who do it grow faster than those who do not. We offer this as a candidate explanation for observed cooperation in rudimentary environments, and as a behavioural baseline for cooperation more generally. This article is part of the theme issue ‘Emergent phenomena in complex physical and socio-technical systems: from cells to societies’.
- 23 May 2022
- Published by The Royal Society in Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences
- Vol. 380 (2227) , 20200425
Open AccessThe ergodicity solution of the cooperation puzzle
When two entities cooperate by sharing resources, one relinquishes something of value to the other. This apparent altruism is frequently observed in nature. Why? Classical treatments assume circumstances where combining resources creates an immediate benefit, e.g. through complementarity or thresholds. Here we ask whether cooperation is predictable without such circumstances. We study a model in which resources self-multiply with fluctuations, a null model of a range of phenomena from viral spread to financial investment. Two fundamental growth rates exist: the ensemble-average growth rate, achieved by the average resources of a large population; and the time-average growth rate, achieved by individual resources over a long time. As a consequence of non-ergodicity, the latter is lower than the former by a term which depends on fluctuation size. Repeated pooling and sharing of resources reduces the effective size of fluctuations and increases the time-average growth rate, which approaches the ensemble-average growth rate in the many-cooperator limit. Therefore, cooperation is advantageous in our model for the simple reason that those who do it grow faster than those who do not. We offer this as a candidate explanation for observed cooperation in rudimentary environments, and as a behavioural baseline for cooperation more generally. This article is part of the theme issue ‘Emergent phenomena in complex physical and socio-technical systems: from cells to societies’.
- 1 December 2021
- Published by Institute for Operations Research and the Management Sciences (INFORMS) in Decision Analysis
- Vol. 18 (4) , 257-272
Microfoundations of Discounting
An important question in economics is how people choose between different payments in the future. The classical normative model predicts that a decision maker discounts a later payment relative to an earlier one by an exponential function of the time between them. Descriptive models use nonexponential functions to fit observed behavioral phenomena, such as preference reversal. Here we propose a model of discounting, consistent with standard axioms of choice, in which decision makers maximize the growth rate of their wealth. Four specifications of the model produce four forms of discounting—no discounting, exponential discounting, hyperbolic discounting, and a hybrid of exponential and hyperbolic discounting—two of which predict preference reversal. Our model requires no assumption of behavioral bias or payment risk.
- 1 December 2021
- Published by Institute for Operations Research and the Management Sciences (INFORMS) in Decision Analysis
- Vol. 18 (4) , 257-272
Microfoundations of Discounting
An important question in economics is how people choose between different payments in the future. The classical normative model predicts that a decision maker discounts a later payment relative to an earlier one by an exponential function of the time between them. Descriptive models use nonexponential functions to fit observed behavioral phenomena, such as preference reversal. Here we propose a model of discounting, consistent with standard axioms of choice, in which decision makers maximize the growth rate of their wealth. Four specifications of the model produce four forms of discounting—no discounting, exponential discounting, hyperbolic discounting, and a hybrid of exponential and hyperbolic discounting—two of which predict preference reversal. Our model requires no assumption of behavioral bias or payment risk.
- Open Access
The time interpretation of expected utility theory
Ergodicity economics is a new branch of economic theory that notes the conceptual difference between time averages and expectation values, which coincide only for ergodic observables. It postulates that individual agents maximise the time average growth rate of wealth, known widely as growth optimality. This contrasts with the dominant behavioural model in economics, expected utility theory, in which agents maximise expectation values of changes in psychologically transformed wealth. Historically, growth optimality was explored for additive and multiplicative gambles. Here we apply it to a general class of wealth dynamics, extending the range of economic situations where it may be used. Moreover, we show a correspondence between growth optimality and expected utility theory, in which the ergodicity transformation in the former is identified as the utility function in the latter. This correspondence offers a theoretical basis for choosing utility functions and predicts that wealth dynamics are strong determinants of risk preferences.
Open AccessThe time interpretation of expected utility theory
Ergodicity economics is a new branch of economic theory that notes the conceptual difference between time averages and expectation values, which coincide only for ergodic observables. It postulates that individual agents maximise the time average growth rate of wealth, known widely as growth optimality. This contrasts with the dominant behavioural model in economics, expected utility theory, in which agents maximise expectation values of changes in psychologically transformed wealth. Historically, growth optimality was explored for additive and multiplicative gambles. Here we apply it to a general class of wealth dynamics, extending the range of economic situations where it may be used. Moreover, we show a correspondence between growth optimality and expected utility theory, in which the ergodicity transformation in the former is identified as the utility function in the latter. This correspondence offers a theoretical basis for choosing utility functions and predicts that wealth dynamics are strong determinants of risk preferences.
- Open Access
Leverage efficiency
Peters (2011a) defined an optimal leverage which maximizes the time-average growth rate of an investment held at constant leverage. It was hypothesized that this optimal leverage is attracted to 1, such that, e.g., leveraging an investment in the market portfolio cannot yield long-term outperformance. This places a strong constraint on the stochastic properties of prices of traded assets, which we call "leverage efficiency." Market conditions that deviate from leverage efficiency are unstable and may create leverage-driven bubbles. Here we expand on the hypothesis and its implications. These include a theory of noise that explains how systemic stability rules out smooth price changes at any pricing frequency; a resolution of the so-called equity premium puzzle; a protocol for central bank interest rate setting to avoid leverage-driven price instabilities; and a method for detecting fraudulent investment schemes by exploiting differences between the stochastic properties of their prices and those of legitimately-traded assets. To submit the hypothesis to a rigorous test we choose price data from different assets: the S&P500 index, Bitcoin, Berkshire Hathaway Inc., and Bernard L. Madoff Investment Securities LLC. Analysis of these data supports the hypothesis.
Open AccessLeverage efficiency
Peters (2011a) defined an optimal leverage which maximizes the time-average growth rate of an investment held at constant leverage. It was hypothesized that this optimal leverage is attracted to 1, such that, e.g., leveraging an investment in the market portfolio cannot yield long-term outperformance. This places a strong constraint on the stochastic properties of prices of traded assets, which we call "leverage efficiency." Market conditions that deviate from leverage efficiency are unstable and may create leverage-driven bubbles. Here we expand on the hypothesis and its implications. These include a theory of noise that explains how systemic stability rules out smooth price changes at any pricing frequency; a resolution of the so-called equity premium puzzle; a protocol for central bank interest rate setting to avoid leverage-driven price instabilities; and a method for detecting fraudulent investment schemes by exploiting differences between the stochastic properties of their prices and those of legitimately-traded assets. To submit the hypothesis to a rigorous test we choose price data from different assets: the S&P500 index, Bitcoin, Berkshire Hathaway Inc., and Bernard L. Madoff Investment Securities LLC. Analysis of these data supports the hypothesis.
- Open Access
What are we weighting for? A mechanistic model for probability weighting
Behavioural economics provides labels for patterns in human economic behaviour. Probability weighting is one such label. It expresses a mismatch between probabilities used in a formal model of a decision (i.e. model parameters) and probabilities inferred from real people's decisions (the same parameters estimated empirically). The inferred probabilities are called "decision weights." It is considered a robust experimental finding that decision weights are higher than probabilities for rare events, and (necessarily, through normalisation) lower than probabilities for common events. Typically this is presented as a cognitive bias, i.e. an error of judgement by the person. Here we point out that the same observation can be described differently: broadly speaking, probability weighting means that a decision maker has greater uncertainty about the world than the observer. We offer a plausible mechanism whereby such differences in uncertainty arise naturally: when a decision maker must estimate probabilities as frequencies in a time series while the observer knows them a priori. This suggests an alternative presentation of probability weighting as a principled response by a decision maker to uncertainties unaccounted for in an observer's model.
Open AccessWhat are we weighting for? A mechanistic model for probability weighting
Behavioural economics provides labels for patterns in human economic behaviour. Probability weighting is one such label. It expresses a mismatch between probabilities used in a formal model of a decision (i.e. model parameters) and probabilities inferred from real people's decisions (the same parameters estimated empirically). The inferred probabilities are called "decision weights." It is considered a robust experimental finding that decision weights are higher than probabilities for rare events, and (necessarily, through normalisation) lower than probabilities for common events. Typically this is presented as a cognitive bias, i.e. an error of judgement by the person. Here we point out that the same observation can be described differently: broadly speaking, probability weighting means that a decision maker has greater uncertainty about the world than the observer. We offer a plausible mechanism whereby such differences in uncertainty arise naturally: when a decision maker must estimate probabilities as frequencies in a time series while the observer knows them a priori. This suggests an alternative presentation of probability weighting as a principled response by a decision maker to uncertainties unaccounted for in an observer's model.
- Open Access
Microfoundations of Discounting
An important question in economics is how people choose between different payments in the future. The classical normative model predicts that a decision maker discounts a later payment relative to an earlier one by an exponential function of the time between them. Descriptive models use non-exponential functions to fit observed behavioral phenomena, such as preference reversal. Here we propose a model of discounting, consistent with standard axioms of choice, in which decision makers maximize the growth rate of their wealth. Four specifications of the model produce four forms of discounting -- no discounting, exponential, hyperbolic, and a hybrid of exponential and hyperbolic -- two of which predict preference reversal. Our model requires no assumption of behavioral bias or payment risk.
Open AccessMicrofoundations of Discounting
An important question in economics is how people choose between different payments in the future. The classical normative model predicts that a decision maker discounts a later payment relative to an earlier one by an exponential function of the time between them. Descriptive models use non-exponential functions to fit observed behavioral phenomena, such as preference reversal. Here we propose a model of discounting, consistent with standard axioms of choice, in which decision makers maximize the growth rate of their wealth. Four specifications of the model produce four forms of discounting -- no discounting, exponential, hyperbolic, and a hybrid of exponential and hyperbolic -- two of which predict preference reversal. Our model requires no assumption of behavioral bias or payment risk.
- Open Access
- 6 December 2019
- Published by Springer Nature in Nature Physics
- Vol. 16 (1) , 111
Author Correction: The ergodicity problem in economics
- 6 December 2019
- Published by Springer Nature in Nature Physics
- Vol. 16 (1) , 111
Open AccessAuthor Correction: The ergodicity problem in economics
- Open Access
- 2 December 2019
- Published by Springer Nature in Nature Physics
- Vol. 15 (12) , 1216-1221
The ergodicity problem in economics
- 2 December 2019
- Published by Springer Nature in Nature Physics
- Vol. 15 (12) , 1216-1221
Open AccessThe ergodicity problem in economics