Jean-Pierre Fouque

Mining blocks in a blockchain using the \textit{Proof-of-Work} consensus protocol involves significant risk, as network participants face continuous operational costs while earning infrequent capital gains upon successfully mining a block. A common risk mitigation strategy is to join a mining pool, which combines the computing resources of multiple miners to provide a more stable income. This article examines a Pay-per-Share (PPS) reward system, where the pool manager can adjust both the share difficulty and the management fee. Using a simplified wealth model for miners, we explore how miners should allocate their computing resources among different mining pools, considering the trade-off between risk transfer to the manager and management fees.

We establish the convergence of the deep actor-critic reinforcement learning algorithm presented in [Angiuli et al., 2023a] in the setting of continuous state and action spaces with an infinite discrete-time horizon. This algorithm provides solutions to Mean Field Game (MFG) or Mean Field Control (MFC) problems depending on the ratio between two learning rates: one for the value function and the other for the mean field term. In the MFC case, to rigorously identify the limit, we introduce a discretization of the state and action spaces, following the approach used in the finite-space case in [Angiuli et al., 2023b]. The convergence proofs rely on a generalization of the two-timescale framework introduced in [Borkar, 1997]. We further extend our convergence results to Mean Field Control Games, which involve locally cooperative and globally competitive populations. Finally, we present numerical experiments for linear-quadratic problems in one and two dimensions, for which explicit solutions are available.

We explicitly connect (discrete-time) quantum walks on Z with a four-state Markov additive process via a Feynman-type formula (2.5). Using this representation, we derive a relation between the spectral decomposition of the Markov additive process and the limiting density of the homogeneous quantum walk. In addition, we consider a space-time rescaling of quantum walks, which leads to a system of quantum transport PDEs in continuous time and space with a phase interaction term. Our probabilistic representation for this type of PDE offers an efficient Monte Carlo computational technique.

We establish the convergence of the unified two-timescale Reinforcement Learning (RL) algorithm presented in a previous work by Angiuli et al. This algorithm provides solutions to Mean Field Game (MFG) or Mean Field Control (MFC) problems depending on the ratio of two learning rates, one for the value function and the other for the mean field term. Our proof of convergence highlights the fact that in the case of MFC several mean field distributions need to be updated and for this reason we present two separate algorithms, one for MFG and one for MFC. We focus on a setting with finite state and action spaces, discrete time and infinite horizon. The proofs of convergence rely on a generalization of the two-timescale approach of Borkar. The accuracy of approximation to the true solutions depends on the smoothing of the policies. We provide a numerical example illustrating the convergence.

We present a new combined \textit{mean field control game} (MFCG) problem which can be interpreted as a competitive game between collaborating groups and its solution as a Nash equilibrium between groups. Players coordinate their strategies within each group. An example is a modification of the classical trader's problem. Groups of traders maximize their wealth. They face cost for their transactions, for their own terminal positions, and for the average holding within their group. The asset price is impacted by the trades of all agents. We propose a three-timescale reinforcement learning algorithm to approximate the solution of such MFCG problems. We test the algorithm on benchmark linear-quadratic specifications for which we provide analytic solutions.