Nils Löhndorf

For many developing countries, COVID‐19 vaccination roll‐out programs are not only slow but vaccination centers are also exposed to the risk of natural disaster, like flooding, which may slow down vaccination progress even further. Policy‐makers in developing countries therefore seek to implement strategies that hedge against distribution risk in order for vaccination campaigns to run smoothly and without delays. We propose a stochastic‐dynamic facility location model that allows policy‐makers to choose vaccination facilities while accounting for possible facility failure. The model is a multi‐stage stochastic variant of the classic facility location problem where disruption risk is modelled as a binary multivariate random process–a problem class that has not yet been studied in the extant literature. To solve the problem, we propose a novel approximate dynamic programming algorithm which trains the shadow price of opening a flood‐prone facility on historical data, thereby alleviating the need to fit a stochastic model. We trained the model using rainfall data provided by the local government of several major cities in the Philippines which are exposed to multiple flooding events per year. Numerical results demonstrate that the solution approach yields approximately 30%–40% lower cost than a baseline approach that does not consider the risk of flooding. Recommendations based on this model were implemented following a collaboration with two large cities in the Philippines which are exposed to multiple flooding events per year.

We consider the stochastic dual dynamic programming (SDDP) algorithm - a widely employed algorithm applied to multistage stochastic programming - and propose a variant using experience replay - a batch learning technique from reinforcement learning. To connect SDDP with reinforcement learning, we cast SDDP as a Q-learning algorithm and describe its application in both risk-neutral and risk-averse settings. We demonstrate the superiority of the algorithm over conventional SDDP by benchmarking it against PSR's SDDP software using a large-scale instance of the long-term planning problem of inter-connected hydropower plants in Colombia. We find that SDDP with batch learning is able to produce tighter optimality gaps in a shorter amount of time than conventional SDDP. We also find that batch learning improves the parallel efficiency of SDDP backward passes.

Grid energy storage plays a key role in making carbon-free, renewable energy production a reality. Yet, when it comes to maximizing profit, owners of storage assets still struggle with coordinating their trading activities across time because of the complex nature of multisettlement electricity markets. In “Coordination of Multimarket Bidding of Grid-Energy Storage,” Nils Löhndorf and David Wozabal propose a multistage stochastic programming model for market-oriented optimization of energy storage. To calculate lower and upper bounds on optimal values, they develop novel methods for scenario-tree generation and information relaxation. They show that a coordinated policy that reserves capacity for the short-term markets is optimal and that the gap to a sequential policy increases with short-term price volatility and market liquidity. The authors find that coordination is beneficial for all considered asset types and that flexible storages with high price impact benefit most. Their findings inform storage owners which markets contribute most value, how to organize trading across time, and how to calculate optimal bidding strategies.