OPOle Peters

Ole Peters

*This is an algorithmically generated author record based on publicly available information.

Ole Peters has a total of 40 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • Open Access

    Non-ergodicity in reinforcement learning: robustness via ergodicity transformations

    by Dominik Baumann, Erfaun Noorani, James Price, Ole Peters, Colm Connaughton, Thomas B. Schön

    Envisioned application areas for reinforcement learning (RL) include autonomous driving, precision agriculture, and finance, which all require RL agents to make decisions in the real world. A significant challenge hindering the adoption of RL methods in these domains is the non-robustness of conventional algorithms. In this paper, we argue that a fundamental issue contributing to this lack of robustness lies in the focus on the expected value of the return as the sole ``correct'' optimization objective. The expected value is the average over the statistical ensemble of infinitely many trajectories. For non-ergodic returns, this average differs from the average over a single but infinitely long trajectory. Consequently, optimizing the expected value can lead to policies that yield exceptionally high returns with probability zero but almost surely result in catastrophic outcomes. This problem can be circumvented by transforming the time series of collected returns into one with ergodic increments. This transformation enables learning robust policies by optimizing the long-term return for individual agents rather than the average across infinitely many trajectories. We propose an algorithm for learning ergodicity transformations from data and demonstrate its effectiveness in an instructive, non-ergodic environment and on standard RL benchmarks.

  • Open Access

    The ergodicity solution of the cooperation puzzle

    by Ole Peters, Alexander Adamou

    When two entities cooperate by sharing resources, one relinquishes something of value to the other. This apparent altruism is frequently observed in nature. Why? Classical treatments assume circumstances where combining resources creates an immediate benefit, e.g. through complementarity or thresholds. Here we ask whether cooperation is predictable without such circumstances. We study a model in which resources self-multiply with fluctuations, a null model of a range of phenomena from viral spread to financial investment. Two fundamental growth rates exist: the ensemble-average growth rate, achieved by the average resources of a large population; and the time-average growth rate, achieved by individual resources over a long time. As a consequence of non-ergodicity, the latter is lower than the former by a term which depends on fluctuation size. Repeated pooling and sharing of resources reduces the effective size of fluctuations and increases the time-average growth rate, which approaches the ensemble-average growth rate in the many-cooperator limit. Therefore, cooperation is advantageous in our model for the simple reason that those who do it grow faster than those who do not. We offer this as a candidate explanation for observed cooperation in rudimentary environments, and as a behavioural baseline for cooperation more generally. This article is part of the theme issue ‘Emergent phenomena in complex physical and socio-technical systems: from cells to societies’.


  • Microfoundations of Discounting

    by Alexander Adamou, Yonatan Berman, Diomides Mavroyiannis, Ole Peters

    An important question in economics is how people choose between different payments in the future. The classical normative model predicts that a decision maker discounts a later payment relative to an earlier one by an exponential function of the time between them. Descriptive models use nonexponential functions to fit observed behavioral phenomena, such as preference reversal. Here we propose a model of discounting, consistent with standard axioms of choice, in which decision makers maximize the growth rate of their wealth. Four specifications of the model produce four forms of discounting—no discounting, exponential discounting, hyperbolic discounting, and a hybrid of exponential and hyperbolic discounting—two of which predict preference reversal. Our model requires no assumption of behavioral bias or payment risk.


  • Open Access

    The time interpretation of expected utility theory

    by Ole Peters, Alexander Adamou

    Ergodicity economics is a new branch of economic theory that notes the conceptual difference between time averages and expectation values, which coincide only for ergodic observables. It postulates that individual agents maximise the time average growth rate of wealth, known widely as growth optimality. This contrasts with the dominant behavioural model in economics, expected utility theory, in which agents maximise expectation values of changes in psychologically transformed wealth. Historically, growth optimality was explored for additive and multiplicative gambles. Here we apply it to a general class of wealth dynamics, extending the range of economic situations where it may be used. Moreover, we show a correspondence between growth optimality and expected utility theory, in which the ergodicity transformation in the former is identified as the utility function in the latter. This correspondence offers a theoretical basis for choosing utility functions and predicts that wealth dynamics are strong determinants of risk preferences.

  • Open Access

    Leverage efficiency

    by Ole Peters, Alexander Adamou

    Peters (2011a) defined an optimal leverage which maximizes the time-average growth rate of an investment held at constant leverage. It was hypothesized that this optimal leverage is attracted to 1, such that, e.g., leveraging an investment in the market portfolio cannot yield long-term outperformance. This places a strong constraint on the stochastic properties of prices of traded assets, which we call "leverage efficiency." Market conditions that deviate from leverage efficiency are unstable and may create leverage-driven bubbles. Here we expand on the hypothesis and its implications. These include a theory of noise that explains how systemic stability rules out smooth price changes at any pricing frequency; a resolution of the so-called equity premium puzzle; a protocol for central bank interest rate setting to avoid leverage-driven price instabilities; and a method for detecting fraudulent investment schemes by exploiting differences between the stochastic properties of their prices and those of legitimately-traded assets. To submit the hypothesis to a rigorous test we choose price data from different assets: the S&P500 index, Bitcoin, Berkshire Hathaway Inc., and Bernard L. Madoff Investment Securities LLC. Analysis of these data supports the hypothesis.

  • Open Access

    What are we weighting for? A mechanistic model for probability weighting

    by Ole Peters, Alexander Adamou, Mark Kirstein, Yonatan Berman

    Behavioural economics provides labels for patterns in human economic behaviour. Probability weighting is one such label. It expresses a mismatch between probabilities used in a formal model of a decision (i.e. model parameters) and probabilities inferred from real people's decisions (the same parameters estimated empirically). The inferred probabilities are called "decision weights." It is considered a robust experimental finding that decision weights are higher than probabilities for rare events, and (necessarily, through normalisation) lower than probabilities for common events. Typically this is presented as a cognitive bias, i.e. an error of judgement by the person. Here we point out that the same observation can be described differently: broadly speaking, probability weighting means that a decision maker has greater uncertainty about the world than the observer. We offer a plausible mechanism whereby such differences in uncertainty arise naturally: when a decision maker must estimate probabilities as frequencies in a time series while the observer knows them a priori. This suggests an alternative presentation of probability weighting as a principled response by a decision maker to uncertainties unaccounted for in an observer's model.

  • Open Access

    Microfoundations of Discounting

    by Alexander T. I. Adamou, Yonatan Berman, Diomides P. Mavroyiannis, Ole B. Peters

    An important question in economics is how people choose between different payments in the future. The classical normative model predicts that a decision maker discounts a later payment relative to an earlier one by an exponential function of the time between them. Descriptive models use non-exponential functions to fit observed behavioral phenomena, such as preference reversal. Here we propose a model of discounting, consistent with standard axioms of choice, in which decision makers maximize the growth rate of their wealth. Four specifications of the model produce four forms of discounting -- no discounting, exponential, hyperbolic, and a hybrid of exponential and hyperbolic -- two of which predict preference reversal. Our model requires no assumption of behavioral bias or payment risk.

  • Open Access

    The ergodicity problem in economics

    by Ole Peters