Backward Approximate Dynamic Programming with Hidden Semi-Markov Stochastic Models in Energy Storage Optimization
Preprint
Abstract
We consider an energy storage problem involving a wind farm with a forecasted power output, a stochastic load, an energy storage device, and a connection to the larger power grid with stochastic prices. Electricity prices and wind power forecast errors are modeled using a novel hidden semi-Markov model that accurately replicates not just the distribution of the errors, but also crossing times, capturing the amount of time each process stays above or below some benchmark such as the forecast. This is an important property of stochastic processes involved in storage problems. We show that we achieve more robust solutions using this model than when more common stochastic models are considered. The new model introduces some additional complexity to the problem as its information states are partially hidden, forming a partially observable Markov decision process. We derive a near-optimal time-dependent policy using backward approximate dynamic programming, which overcomes the computational hurdles of classical (exact) backward dynamic programming, with higher quality solutions than the more familiar forward approximate dynamic programming methods.Keywords
All Related Versions
This publication has 27 references indexed in Scilit:
- The challenge of integrating offshore wind power in the U.S. electric grid. Part II: Simulation of electricity market operationsRenewable Energy, 2017
- Reducing the computational effort of optimal process controllers for continuous state spaces by using incremental learning and post-decision state formulationsJournal of Process Control, 2014
- A stochastic dynamic programming model for co-optimization of distributed energy storageEnergy Systems, 2013
- The Challenges of Dynamic Programming2011
- Analysis of stochastic dual dynamic programming methodEuropean Journal of Operational Research, 2011
- Optimal day-ahead trading and storage of renewable energies—an approximate dynamic programming approachEnergy Systems, 2010
- The value of compressed air energy storage with wind in transmission-constrained electric power systemsEnergy Policy, 2009
- On the convergence of stochastic dual dynamic programming and related methodsOperations Research Letters, 2008
- Cut sharing for multistage stochastic linear programs with interstage dependencyMathematical Programming, 1996
- Multi-stage stochastic optimization applied to energy planningMathematical Programming, 1991