JNJuliana Nascimento

Juliana Nascimento

*This is an algorithmically generated author record based on publicly available information.

Current Organizations / Institutes

The organizations/institutes from the scholar's latest article


Juliana Nascimento has a total of 3 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • Open Access

    Risk Directed Importance Sampling in Stochastic Dual Dynamic Programming with Hidden Markov Models for Grid Level Energy Storage

    by Joseph L. Durante, Juliana Nascimento, Warren B. Powell

    Power systems that need to integrate renewables at a large scale must account for the high levels of uncertainty introduced by these power sources. This can be accomplished with a system of many distributed grid-level storage devices. However, developing a cost-effective and robust control policy in this setting is a challenge due to the high dimensionality of the resource state and the highly volatile stochastic processes involved. We first model the problem using a carefully calibrated power grid model and a specialized hidden Markov stochastic model for wind power which replicates crossing times. We then base our control policy on a variant of stochastic dual dynamic programming, an algorithm well suited for certain high dimensional control problems, that is modified to accommodate hidden Markov uncertainty in the stochastics. However, the algorithm may be impractical to use as it exhibits relatively slow convergence. To accelerate the algorithm, we apply both quadratic regularization and a risk-directed importance sampling technique for sampling the outcome space at each time step in the backward pass of the algorithm. We show that the resulting policies are more robust than those developed using classical SDDP modeling assumptions and algorithms.

  • Open Access

    Backward Approximate Dynamic Programming with Hidden Semi-Markov Stochastic Models in Energy Storage Optimization

    by Joseph L. Durante, Juliana Nascimento, Warren B. Powell

    We consider an energy storage problem involving a wind farm with a forecasted power output, a stochastic load, an energy storage device, and a connection to the larger power grid with stochastic prices. Electricity prices and wind power forecast errors are modeled using a novel hidden semi-Markov model that accurately replicates not just the distribution of the errors, but also crossing times, capturing the amount of time each process stays above or below some benchmark such as the forecast. This is an important property of stochastic processes involved in storage problems. We show that we achieve more robust solutions using this model than when more common stochastic models are considered. The new model introduces some additional complexity to the problem as its information states are partially hidden, forming a partially observable Markov decision process. We derive a near-optimal time-dependent policy using backward approximate dynamic programming, which overcomes the computational hurdles of classical (exact) backward dynamic programming, with higher quality solutions than the more familiar forward approximate dynamic programming methods.

  • Open Access

    Reinforcement Learning for Dynamic Bidding in Truckload Markets: an Application to Large-Scale Fleet Management with Advance Commitments

    by Yingfei Wang, Juliana Martins Do Nascimento, Warren Powell

    Truckload brokerages, a $100 billion/year industry in the U.S., plays the critical role of matching shippers with carriers, often to move loads several days into the future. Brokerages not only have to find companies that will agree to move a load, the brokerage often has to find a price that both the shipper and carrier will agree to. The price not only varies by shipper and carrier, but also by the traffic lanes and other variables such as commodity type. Brokerages have to learn about shipper and carrier response functions by offering a price and observing whether each accepts the quote. We propose a knowledge gradient policy with bootstrap aggregation for high-dimensional contextual settings to guide price experimentation by maximizing the value of information. The learning policy is tested using a carefully calibrated fleet simulator that includes a stochastic lookahead policy that simulates fleet movements, as well as the stochastic modeling of driver assignments and the carrier's load commitment policies with advance booking.

  • An Optimal ADP Algorithm for a High-Dimensional Stochastic Control Problem

    by Juliana Nascimento, Warren Powell

    We propose a provably optimal approximate dynamic programming algorithm for a class of multistage stochastic problems, taking into account that the probability distribution of the underlying stochastic process is not known and the state space is too large to be explored entirely. The algorithm and its proof of convergence rely on the fact that the optimal value functions of the problems within the problem class are concave and piecewise linear. The algorithm is a combination of Monte Carlo simulation, pure exploitation, stochastic approximation and a projection operation. Several applications, in areas like energy, control, inventory and finance, fall under the framework