Assessment of Reinforcement Learning Algorithms for Nuclear Power Plant Fuel Optimization
Preprint
Abstract
The nuclear fuel loading pattern optimization problem has been studied since the dawn of the commercial nuclear energy industry. It is characterized by multiple objectives and constraints, with a very high number of candidate patterns, which makes it impossible to solve explicitly. Stochastic optimization methodologies are used by different nuclear utilities and vendors to perform fuel cycle reload design. Nevertheless, hand-designed solutions continue to be the prevalent method in the industry. To improve the state-of-the-art core reload patterns, we aim to create a method as scalable as possible, that agrees with the designer's goal of performance and safety. To help in this task Deep Reinforcement Learning (RL), in particular, Proximal Policy Optimization is leveraged. RL has recently experienced a strong impetus from its successes applied to games. This paper lays out the foundation of this method and proposes to study the behavior of several hyper-parameters that influence the RL algorithm via a multi-measure approach helped with statistical tests. The algorithm is highly dependent on multiple factors such as the shape of the objective function derived for the core design that behaves as a fudge factor that affects the stability of the learning. But also an exploration/exploitation trade-off that manifests through different parameters such as the number of loading patterns seen by the agents per episode, the number of samples collected before a policy update, and an entropy factor that increases the randomness of the policy trained. Experimental results also demonstrate the effectiveness of the method in finding high-quality solutions from scratch within a reasonable amount of time. Future work must include applying the algorithms to wide range of applications and comparing them to state-of-the-art implementation of stochastic optimization methods.Keywords
All Related Versions
This publication has 33 references indexed in Scilit:
- Comparative study of meta-heuristic algorithms for reactor fuel reloading optimization based on the developed BP-ANN calculation methodAnnals of Nuclear Energy, 2022
- Reinforcement learning for combinatorial optimization: A surveyComputers & Operations Research, 2021
- Large-scale design optimisation of boiling water reactor bundles with neuroevolutionAnnals of Nuclear Energy, 2021
- A comparative study on multiobjective metaheuristics for solving constrained in-core fuel management optimisation problemsComputers & Operations Research, 2016
- Quantum evolutionary algorithm and tabu search in pressurized water reactor loading pattern designAnnals of Nuclear Energy, 2016
- Comparison of metaheuristic optimization techniques for BWR fuel reloads pattern designAnnals of Nuclear Energy, 2013
- Sensitivity study on determining an efficient set of fuel assembly parameters in training data for designing of neural networks in hybrid genetic algorithmsAnnals of Nuclear Energy, 2006
- Development of a BWR loading pattern design system based on modified genetic algorithms and knowledgeAnnals of Nuclear Energy, 2004
- Using a multi-state recurrent neural network to optimize loading patterns in BWRsAnnals of Nuclear Energy, 2004
- Upgrading automation for nuclear fuel in-core management: From the symbolic generation of configurations, to the neural adaptation of heuristicsEngineering with Computers, 1997