RCRené Carmona

René Carmona

*This is an algorithmically generated author record based on publicly available information.

Current Organizations / Institutes

The organizations/institutes from the scholar's latest article


René Carmona has a total of 57 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • Reconciling Discrete-Time Mixed Policies and Continuous-Time Relaxed Controls in Reinforcement Learning and Stochastic Control

    by René Carmona, Mathieu Laurière

    Reinforcement learning (RL) is currently one of the most prominent methods for optimizing dynamical systems, with breakthrough results across various fields. The framework is based on the concept of a Markov decision process (MDP), leading to a discrete-time optimal control problem. In the RL literature, such problems are typically formulated and solved using mixed policies, from which random actions are sampled at each time step. Recently, part of the optimal control community has begun investigating continuous-time versions of RL algorithms, replacing MDPs with continuous-time stochastic processes governed by relaxed controls and asserting a full analogy between the two formulations. In this work, we examine the limitations of this analogy and rigorously establish a connection between the two problems in the case where only the drift term of the continuous-time model is controlled. We prove strong convergence of the RL implementation of mixed strategies as the time discretization mesh tends to zero. We also discuss the technical challenges posed by the possible presence of control in the diffusion component of the state.

  • Open Access

    Cost Attribution And Risk-Averse Unit Commitment In Power Grids Using Integrated Gradient

    by Rene Carmona, Ronnie Sircar, Xinshuo Yang

    This paper introduces a novel approach to addressing uncertainty and associated risks in power system management, focusing on deviations between forecasted and actual load demand and renewable power generation. By employing Economic Dispatch with both day-ahead forecasts and actual values, we derive two distinct system costs, revealing the financial risks stemming from uncertainty. We present a numerical algorithm inspired by the Integrated Gradients method to attribute the contribution of stochastic components to the difference in system costs. This method, originally developed for machine learning, facilitates the understanding of individual input features' impact on the model's output prediction. By assigning numeric values to represent the influence of variability on operational costs, our method provides actionable insights for grid management. As an application, we propose a risk-averse unit commitment framework, leveraging our cost attribution algorithm to adjust the capacity of renewable generators, thus mitigating system risk. Simulation results on the RTS-GMLC grid demonstrate the efficacy of our approach in improving grid reliability and reducing operational costs.

  • Open Access

    Particle approximation for conditional control with soft killing

    by René Carmona, Samuel Daudin

    The aim of this paper is to develop a particle approximation for the conditional control problem introduced by P.-L. Lions during his lectures at the Collège de France in November 2016. We focus on a soft killing relaxed version of the problem, which admits a natural counterpart in terms of stochastic optimal control for a large number of interacting particles. Each particle contributes to the overall population cost through a time-evolving weight that depends on the trajectories of all the other particles. Using recently developed techniques for the analysis of the value function associated with the limiting problem, we establish sharp convergence rates as the number of particles tends to infinity.

  • Open Access

    Conditional McKean-Vlasov control

    by René Carmona, Ludovic Tangpi, Kaiwen Zhang

    Conditional McKean-Vlasov control problems involve controlling McKean-Vlasov diffusions where the interaction occurs through the law of the state process conditionally on it staying in a domain. Introduced by Lions in his 2016 lectures at the Collège de France, these problems have notable applications, particularly in systemic risk. We establish well-posedness and provide a general characterization of optimal controls using a new Pontryagin maximum principle in the probabilistic weak formulation. Unlike the classical approach based on forward-backward systems, our results connect the control problem to a generalized McKean-Vlasov backward stochastic differential equation (BSDE). We illustrate our framework with two applications: a version of the Schrödinger problem with killing, and a construction of equilibria in potential mean field games via McKean-Vlasov control.

  • Open Access

    Kuramoto Mean Field Game with Intrinsic Frequencies

    by Rene Carmona, Quentin Cormier, Mete Soner

    This paper studies a mean field game formulation of the classical Kuramoto model for synchronization. Our model captures the diversity within the population by considering random intrinsic frequencies, which allows us to study the impact of this heterogeneity on synchronization patterns and stability. Our findings contribute insights into the interplay between intrinsic frequency diversity and synchronization dynamics, offering a more realistic understanding of complex systems. The proposed framework has broad applications ranging from coupled oscillators in physics to social dynamics, and serves as a valuable tool for studying networks with distributed intrinsic frequencies.

  • Open Access

    Particle Approximation for Conditional Control with Soft Killing

    by Rene Carmona, Samuel Daudin

    The aim of this paper is to develop a particle approximation for the conditional control problem introduced by P.-L. Lions during his lectures at the Collège de France in November 2016. We focus on a \textit{soft killing} relaxed version of the problem, which admits a natural counterpart in terms of stochastic optimal control for a large number of interacting particles. Each particle contributes to the overall population cost through a time-evolving weight that depends on the trajectories of all the other particles. Using recently developed techniques for the analysis of the value function associated with the limiting problem, we establish sharp convergence rates as the number of particles tends to infinity.

  • Open Access

    Maximizing On-Bill Savings through Battery Management Optimization

    by Rene Carmona, Xinshuo Yang, Siddharth Bhela, Claire Zeng

    In many power grids, a large portion of the energy costs for commercial and industrial consumers are set with reference to the coincident peak load, the demand during the maximum system-wide peak, and their own maximum peak load, the non-coincident peak load. Coincident-peak based charges reflect the allocation of infrastructure updates to end-users for increased capacity, the amount the grid can handle, and for improvement of the transmission, the ability to transport energy across the network. Demand charges penalize the stress on the grid caused by each consumer's peak demand. Microgrids with a local generator, controllable loads, and/or a battery technology have the flexibility to cut their peak load contributions and thereby significantly reduce these charges. This paper investigates the optimal planning of microgrid technology for electricity bill reduction. The specificity of our approach is the leveraging of a scenario generator engine to incorporate probability estimates of coincident peaks and non-coincident peaks into the optimization problem.

  • Open Access

    Leveraging the Turnpike Effect for Mean Field Games Numerics

    by René A. Carmona, Claire Zeng

    Recently, a deep-learning algorithm referred to as Deep Galerkin Method (DGM), has gained a lot of attention among those trying to solve numerically Mean Field Games with finite horizon, even if the performance seems to be decreasing significantly with increasing horizon. On the other hand, it has been proven that some specific classes of Mean Field Games enjoy some form of the turnpike property identified over seven decades ago by economists. The gist of this phenomenon is a proof that the solution of an optimal control problem over a long time interval spends most of its time near the stationary solution of the ergodic version of the corresponding infinite horizon optimization problem. After reviewing the implementation of DGM for finite horizon Mean Field Games, we introduce a “turnpike-accelerated” version that incorporates the turnpike estimates in the loss function to be optimized, and we perform a comparative numerical analysis to show the advantages of this accelerated version over the baseline DGM algorithm. We demonstrate on some of the Mean Field Game models with local-couplings known to have the turnpike property, as well as a new class of linear-quadratic models for which we derive explicit turnpike estimates.


  • Open Access

    Leveraging the turnpike effect for Mean Field Games numerics

    by René Carmona, Claire Zeng

    Recently, a deep-learning algorithm referred to as Deep Galerkin Method (DGM), has gained a lot of attention among those trying to solve numerically Mean Field Games with finite horizon, even if the performance seems to be decreasing significantly with increasing horizon. On the other hand, it has been proven that some specific classes of Mean Field Games enjoy some form of the turnpike property identified over seven decades ago by economists. The gist of this phenomenon is a proof that the solution of an optimal control problem over a long time interval spends most of its time near the stationary solution of the ergodic solution of the corresponding infinite horizon optimization problem. After reviewing the implementation of DGM for finite horizon Mean Field Games, we introduce a ``turnpike-accelerated'' version that incorporates the turnpike estimates in the loss function to be optimized, and we perform a comparative numerical analysis to show the advantages of this accelerated version over the baseline DGM algorithm. We demonstrate on some of the Mean Field Game models with local-couplings known to have the turnpike property, as well as a new class of linear-quadratic models for which we derive explicit turnpike estimates.