René Carmona
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
René Carmona has a total of 57 co-authors. Here are the top co-authors by number of shared publications:
- Mathieu Laurière
Shanghai Center for Data Science NYU-ECNU Institute of Mathematical Sciences NYU Shanghai NYU Shanghai Shanghai People's Republic of China
10 shared publications - Gokce Dayanikli
Department of Statistics University of Illinois Urbana-Champaign 61820 Champaign IL USA
7 shared publications - François Delarue
Laboratoire J.-A. Dieudonné, Université Côte d’Azur, Cedex 02, France
5 shared publications - Alexander Aurell
Department of Operations Research and Financial Engineering, Princeton University, Princeton, USA
4 shared publications - Daniel Schwarz
University College London
3 shared publications - Michael Coulon
ORFE Department, Princeton University, Princeton, USA
3 shared publications
Latest Publications
- 12 November 2026
- Published by Society for Industrial & Applied Mathematics (SIAM) in SIAM Review
- Vol. 68 (4) , 823-849
Reconciling Discrete-Time Mixed Policies and Continuous-Time Relaxed Controls in Reinforcement Learning and Stochastic Control
Reinforcement learning (RL) is currently one of the most prominent methods for optimizing dynamical systems, with breakthrough results across various fields. The framework is based on the concept of a Markov decision process (MDP), leading to a discrete-time optimal control problem. In the RL literature, such problems are typically formulated and solved using mixed policies, from which random actions are sampled at each time step. Recently, part of the optimal control community has begun investigating continuous-time versions of RL algorithms, replacing MDPs with continuous-time stochastic processes governed by relaxed controls and asserting a full analogy between the two formulations. In this work, we examine the limitations of this analogy and rigorously establish a connection between the two problems in the case where only the drift term of the continuous-time model is controlled. We prove strong convergence of the RL implementation of mixed strategies as the time discretization mesh tends to zero. We also discuss the technical challenges posed by the possible presence of control in the diffusion component of the state.
- 12 November 2026
- Published by Society for Industrial & Applied Mathematics (SIAM) in SIAM Review
- Vol. 68 (4) , 823-849
Reconciling Discrete-Time Mixed Policies and Continuous-Time Relaxed Controls in Reinforcement Learning and Stochastic Control
Reinforcement learning (RL) is currently one of the most prominent methods for optimizing dynamical systems, with breakthrough results across various fields. The framework is based on the concept of a Markov decision process (MDP), leading to a discrete-time optimal control problem. In the RL literature, such problems are typically formulated and solved using mixed policies, from which random actions are sampled at each time step. Recently, part of the optimal control community has begun investigating continuous-time versions of RL algorithms, replacing MDPs with continuous-time stochastic processes governed by relaxed controls and asserting a full analogy between the two formulations. In this work, we examine the limitations of this analogy and rigorously establish a connection between the two problems in the case where only the drift term of the continuous-time model is controlled. We prove strong convergence of the RL implementation of mixed strategies as the time discretization mesh tends to zero. We also discuss the technical challenges posed by the possible presence of control in the diffusion component of the state.
- Open Access
- 27 June 2026
- Published by Elsevier in SSRN Electronic Journal
Cost Attribution And Risk-Averse Unit Commitment In Power Grids Using Integrated Gradient
This paper introduces a novel approach to addressing uncertainty and associated risks in power system management, focusing on deviations between forecasted and actual load demand and renewable power generation. By employing Economic Dispatch with both day-ahead forecasts and actual values, we derive two distinct system costs, revealing the financial risks stemming from uncertainty. We present a numerical algorithm inspired by the Integrated Gradients method to attribute the contribution of stochastic components to the difference in system costs. This method, originally developed for machine learning, facilitates the understanding of individual input features' impact on the model's output prediction. By assigning numeric values to represent the influence of variability on operational costs, our method provides actionable insights for grid management. As an application, we propose a risk-averse unit commitment framework, leveraging our cost attribution algorithm to adjust the capacity of renewable generators, thus mitigating system risk. Simulation results on the RTS-GMLC grid demonstrate the efficacy of our approach in improving grid reliability and reducing operational costs.
- 27 June 2026
- Published by Elsevier in SSRN Electronic Journal
Open AccessCost Attribution And Risk-Averse Unit Commitment In Power Grids Using Integrated Gradient
This paper introduces a novel approach to addressing uncertainty and associated risks in power system management, focusing on deviations between forecasted and actual load demand and renewable power generation. By employing Economic Dispatch with both day-ahead forecasts and actual values, we derive two distinct system costs, revealing the financial risks stemming from uncertainty. We present a numerical algorithm inspired by the Integrated Gradients method to attribute the contribution of stochastic components to the difference in system costs. This method, originally developed for machine learning, facilitates the understanding of individual input features' impact on the model's output prediction. By assigning numeric values to represent the influence of variability on operational costs, our method provides actionable insights for grid management. As an application, we propose a risk-averse unit commitment framework, leveraging our cost attribution algorithm to adjust the capacity of renewable generators, thus mitigating system risk. Simulation results on the RTS-GMLC grid demonstrate the efficacy of our approach in improving grid reliability and reducing operational costs.
- Open Access
- 10 June 2026
- Published by EDP Sciences in ESAIM: Control, Optimisation and Calculus of Variations
- Vol. 32, 62
Particle approximation for conditional control with soft killing
The aim of this paper is to develop a particle approximation for the conditional control problem introduced by P.-L. Lions during his lectures at the Collège de France in November 2016. We focus on a soft killing relaxed version of the problem, which admits a natural counterpart in terms of stochastic optimal control for a large number of interacting particles. Each particle contributes to the overall population cost through a time-evolving weight that depends on the trajectories of all the other particles. Using recently developed techniques for the analysis of the value function associated with the limiting problem, we establish sharp convergence rates as the number of particles tends to infinity.
- 10 June 2026
- Published by EDP Sciences in ESAIM: Control, Optimisation and Calculus of Variations
- Vol. 32, 62
Open AccessParticle approximation for conditional control with soft killing
The aim of this paper is to develop a particle approximation for the conditional control problem introduced by P.-L. Lions during his lectures at the Collège de France in November 2016. We focus on a soft killing relaxed version of the problem, which admits a natural counterpart in terms of stochastic optimal control for a large number of interacting particles. Each particle contributes to the overall population cost through a time-evolving weight that depends on the trajectories of all the other particles. Using recently developed techniques for the analysis of the value function associated with the limiting problem, we establish sharp convergence rates as the number of particles tends to infinity.
- 1 April 2026
- Published by Springer Nature
- p. 311-320
Mimicking and Conditional Control with Hard Killing
- 1 April 2026
- Published by Springer Nature
- p. 311-320
Mimicking and Conditional Control with Hard Killing
- Open Access
Conditional McKean-Vlasov control
Conditional McKean-Vlasov control problems involve controlling McKean-Vlasov diffusions where the interaction occurs through the law of the state process conditionally on it staying in a domain. Introduced by Lions in his 2016 lectures at the Collège de France, these problems have notable applications, particularly in systemic risk. We establish well-posedness and provide a general characterization of optimal controls using a new Pontryagin maximum principle in the probabilistic weak formulation. Unlike the classical approach based on forward-backward systems, our results connect the control problem to a generalized McKean-Vlasov backward stochastic differential equation (BSDE). We illustrate our framework with two applications: a version of the Schrödinger problem with killing, and a construction of equilibria in potential mean field games via McKean-Vlasov control.
Open AccessConditional McKean-Vlasov control
Conditional McKean-Vlasov control problems involve controlling McKean-Vlasov diffusions where the interaction occurs through the law of the state process conditionally on it staying in a domain. Introduced by Lions in his 2016 lectures at the Collège de France, these problems have notable applications, particularly in systemic risk. We establish well-posedness and provide a general characterization of optimal controls using a new Pontryagin maximum principle in the probabilistic weak formulation. Unlike the classical approach based on forward-backward systems, our results connect the control problem to a generalized McKean-Vlasov backward stochastic differential equation (BSDE). We illustrate our framework with two applications: a version of the Schrödinger problem with killing, and a construction of equilibria in potential mean field games via McKean-Vlasov control.
- Open Access
Kuramoto Mean Field Game with Intrinsic Frequencies
This paper studies a mean field game formulation of the classical Kuramoto model for synchronization. Our model captures the diversity within the population by considering random intrinsic frequencies, which allows us to study the impact of this heterogeneity on synchronization patterns and stability. Our findings contribute insights into the interplay between intrinsic frequency diversity and synchronization dynamics, offering a more realistic understanding of complex systems. The proposed framework has broad applications ranging from coupled oscillators in physics to social dynamics, and serves as a valuable tool for studying networks with distributed intrinsic frequencies.
Open AccessKuramoto Mean Field Game with Intrinsic Frequencies
This paper studies a mean field game formulation of the classical Kuramoto model for synchronization. Our model captures the diversity within the population by considering random intrinsic frequencies, which allows us to study the impact of this heterogeneity on synchronization patterns and stability. Our findings contribute insights into the interplay between intrinsic frequency diversity and synchronization dynamics, offering a more realistic understanding of complex systems. The proposed framework has broad applications ranging from coupled oscillators in physics to social dynamics, and serves as a valuable tool for studying networks with distributed intrinsic frequencies.
- Open Access
Particle Approximation for Conditional Control with Soft Killing
The aim of this paper is to develop a particle approximation for the conditional control problem introduced by P.-L. Lions during his lectures at the Collège de France in November 2016. We focus on a \textit{soft killing} relaxed version of the problem, which admits a natural counterpart in terms of stochastic optimal control for a large number of interacting particles. Each particle contributes to the overall population cost through a time-evolving weight that depends on the trajectories of all the other particles. Using recently developed techniques for the analysis of the value function associated with the limiting problem, we establish sharp convergence rates as the number of particles tends to infinity.
Open AccessParticle Approximation for Conditional Control with Soft Killing
The aim of this paper is to develop a particle approximation for the conditional control problem introduced by P.-L. Lions during his lectures at the Collège de France in November 2016. We focus on a \textit{soft killing} relaxed version of the problem, which admits a natural counterpart in terms of stochastic optimal control for a large number of interacting particles. Each particle contributes to the overall population cost through a time-evolving weight that depends on the trajectories of all the other particles. Using recently developed techniques for the analysis of the value function associated with the limiting problem, we establish sharp convergence rates as the number of particles tends to infinity.
- Open Access
Maximizing On-Bill Savings through Battery Management Optimization
In many power grids, a large portion of the energy costs for commercial and industrial consumers are set with reference to the coincident peak load, the demand during the maximum system-wide peak, and their own maximum peak load, the non-coincident peak load. Coincident-peak based charges reflect the allocation of infrastructure updates to end-users for increased capacity, the amount the grid can handle, and for improvement of the transmission, the ability to transport energy across the network. Demand charges penalize the stress on the grid caused by each consumer's peak demand. Microgrids with a local generator, controllable loads, and/or a battery technology have the flexibility to cut their peak load contributions and thereby significantly reduce these charges. This paper investigates the optimal planning of microgrid technology for electricity bill reduction. The specificity of our approach is the leveraging of a scenario generator engine to incorporate probability estimates of coincident peaks and non-coincident peaks into the optimization problem.
Open AccessMaximizing On-Bill Savings through Battery Management Optimization
In many power grids, a large portion of the energy costs for commercial and industrial consumers are set with reference to the coincident peak load, the demand during the maximum system-wide peak, and their own maximum peak load, the non-coincident peak load. Coincident-peak based charges reflect the allocation of infrastructure updates to end-users for increased capacity, the amount the grid can handle, and for improvement of the transmission, the ability to transport energy across the network. Demand charges penalize the stress on the grid caused by each consumer's peak demand. Microgrids with a local generator, controllable loads, and/or a battery technology have the flexibility to cut their peak load contributions and thereby significantly reduce these charges. This paper investigates the optimal planning of microgrid technology for electricity bill reduction. The specificity of our approach is the leveraging of a scenario generator engine to incorporate probability estimates of coincident peaks and non-coincident peaks into the optimization problem.
- Open Access
- 26 June 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Open Journal of Control Systems
- Vol. 3, 389-404
Leveraging the Turnpike Effect for Mean Field Games Numerics
Recently, a deep-learning algorithm referred to as Deep Galerkin Method (DGM), has gained a lot of attention among those trying to solve numerically Mean Field Games with finite horizon, even if the performance seems to be decreasing significantly with increasing horizon. On the other hand, it has been proven that some specific classes of Mean Field Games enjoy some form of the turnpike property identified over seven decades ago by economists. The gist of this phenomenon is a proof that the solution of an optimal control problem over a long time interval spends most of its time near the stationary solution of the ergodic version of the corresponding infinite horizon optimization problem. After reviewing the implementation of DGM for finite horizon Mean Field Games, we introduce a “turnpike-accelerated” version that incorporates the turnpike estimates in the loss function to be optimized, and we perform a comparative numerical analysis to show the advantages of this accelerated version over the baseline DGM algorithm. We demonstrate on some of the Mean Field Game models with local-couplings known to have the turnpike property, as well as a new class of linear-quadratic models for which we derive explicit turnpike estimates.
- 26 June 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Open Journal of Control Systems
- Vol. 3, 389-404
Open AccessLeveraging the Turnpike Effect for Mean Field Games Numerics
Recently, a deep-learning algorithm referred to as Deep Galerkin Method (DGM), has gained a lot of attention among those trying to solve numerically Mean Field Games with finite horizon, even if the performance seems to be decreasing significantly with increasing horizon. On the other hand, it has been proven that some specific classes of Mean Field Games enjoy some form of the turnpike property identified over seven decades ago by economists. The gist of this phenomenon is a proof that the solution of an optimal control problem over a long time interval spends most of its time near the stationary solution of the ergodic version of the corresponding infinite horizon optimization problem. After reviewing the implementation of DGM for finite horizon Mean Field Games, we introduce a “turnpike-accelerated” version that incorporates the turnpike estimates in the loss function to be optimized, and we perform a comparative numerical analysis to show the advantages of this accelerated version over the baseline DGM algorithm. We demonstrate on some of the Mean Field Game models with local-couplings known to have the turnpike property, as well as a new class of linear-quadratic models for which we derive explicit turnpike estimates.
- Open Access
Leveraging the turnpike effect for Mean Field Games numerics
Recently, a deep-learning algorithm referred to as Deep Galerkin Method (DGM), has gained a lot of attention among those trying to solve numerically Mean Field Games with finite horizon, even if the performance seems to be decreasing significantly with increasing horizon. On the other hand, it has been proven that some specific classes of Mean Field Games enjoy some form of the turnpike property identified over seven decades ago by economists. The gist of this phenomenon is a proof that the solution of an optimal control problem over a long time interval spends most of its time near the stationary solution of the ergodic solution of the corresponding infinite horizon optimization problem. After reviewing the implementation of DGM for finite horizon Mean Field Games, we introduce a ``turnpike-accelerated'' version that incorporates the turnpike estimates in the loss function to be optimized, and we perform a comparative numerical analysis to show the advantages of this accelerated version over the baseline DGM algorithm. We demonstrate on some of the Mean Field Game models with local-couplings known to have the turnpike property, as well as a new class of linear-quadratic models for which we derive explicit turnpike estimates.
Open AccessLeveraging the turnpike effect for Mean Field Games numerics
Recently, a deep-learning algorithm referred to as Deep Galerkin Method (DGM), has gained a lot of attention among those trying to solve numerically Mean Field Games with finite horizon, even if the performance seems to be decreasing significantly with increasing horizon. On the other hand, it has been proven that some specific classes of Mean Field Games enjoy some form of the turnpike property identified over seven decades ago by economists. The gist of this phenomenon is a proof that the solution of an optimal control problem over a long time interval spends most of its time near the stationary solution of the ergodic solution of the corresponding infinite horizon optimization problem. After reviewing the implementation of DGM for finite horizon Mean Field Games, we introduce a ``turnpike-accelerated'' version that incorporates the turnpike estimates in the loss function to be optimized, and we perform a comparative numerical analysis to show the advantages of this accelerated version over the baseline DGM algorithm. We demonstrate on some of the Mean Field Game models with local-couplings known to have the turnpike property, as well as a new class of linear-quadratic models for which we derive explicit turnpike estimates.