Using the proximal policy optimisation algorithm for solving the stochastic capacitated lot sizing problem
Open Access
- 7 April 2022
- journal article
- research article
- Published by Taylor & Francis in International Journal of Production Research
- Vol. 61 (6) , 1955-1978
Abstract
This paper studies the multi-item stochastic capacitated lot-sizing problem with stationary demand to minimise set-up, holding, and backorder costs. This is a common problem in the industry, concerning both inventory management and production planning. We study the applicability of the Proximal Policy Optimisation (PPO) algorithm in this problem, which is a type of Deep Reinforcement Learning (DRL). The problem is modelled as a Markov Decision Process (MDP), which can be solved to optimality in small problem instances by using Dynamic Programming. In these settings, we show that the performance of PPO approaches the optimal solution. For larger problem instances with an increasing number of products, solving to optimality is intractable, and we demonstrate that the PPO solution outperforms the benchmark solution. Several adjustments to the standard PPO algorithm are implemented to make it more scalable to larger problem instances. We show the linear growth in computation time for the algorithm, and present a method for explaining the outcomes of the algorithm. We suggest future research directions that could improve the scalability and explainability of the PPO algorithm.Keywords
This publication has 41 references indexed in Scilit:
- A column generation heuristic for dynamic capacitated lot sizing with random demand under a fill rate constraintOmega, 2011
- Case-based reinforcement learning for dynamic inventory control in a multi-agent supply-chain systemExpert Systems with Applications, 2009
- A reinforcement learning model for supply chain ordering management: An application to the beer gameDecision Support Systems, 2008
- Case-based myopic reinforcement learning for satisfying target service level in supply chainExpert Systems with Applications, 2008
- Modeling industrial lot sizing problems: a reviewInternational Journal of Production Research, 2007
- Multi-item capacitated lot-sizing with demand uncertaintyInternational Journal of Production Research, 2006
- The capacitated lot sizing problem: a review of models and algorithmsOmega, 2003
- AN EXPERIMENTAL STUDY OF THE EFFECTIVENESS OF ROLLING SCHEDULES IN PRODUCTION PLANNINGDecision Sciences, 1977
- The theory of dynamic programmingBulletin of the American Mathematical Society, 1954
- Optimal Inventory PolicyEconometrica, 1951