HMA-PPO for integrated production and preventive maintenance scheduling in distributed flexible job shops with job arrivals
Open Access
- 29 August 2026
- journal article
- Published by Oxford University Press (OUP) in Journal of Computational Design and Engineering
- Vol. 13 (9) , 116-141
Abstract
With the rapid advancement of distributed manufacturing, distributed flexible job shops have become an important production paradigm in high-end equipment manufacturing. However, dynamic job arrivals and preventive maintenance requirements arising from continuous equipment degradation create strong coupling between production scheduling and maintenance decisions, making multi-objective optimization particularly challenging. This study investigates the integrated optimization of production scheduling and preventive maintenance in distributed flexible job shops with dynamic job arrivals, with the objectives of minimizing makespan, total tardiness, and total energy consumption. To solve this problem, a hierarchical multi-agent proximal policy optimization approach, termed HMA-PPO, is proposed. The method establishes a collaborative decision-making framework that involves production agents and maintenance agents, enabling the joint optimization of production scheduling and preventive maintenance. A multi-objective learning mechanism that combines NSGA-II-generated reference solutions with dynamic weights derived from the entropy weight method is further developed to alleviate reward sparsity and improve multi-objective trade-off performance. In addition, a discrete-event simulation environment is developed in FlexSim to model and evaluate the collaborative scheduling process under dynamic job arrivals. Experimental results on generated test instances show that HMA-PPO consistently outperforms PPO, SAC, DDQN, and DQN in terms of convergence quality and Pareto-set diversity, as evaluated using IGD, Nd, C(A,*), and NR.Keywords
Funding Information
- Key Research and Development Program of Henan Province (231111222600)
This publication has 69 references indexed in Scilit:
- MILP modeling and optimization of flexible job shop scheduling problem with preventive maintenanceComputers & Industrial Engineering, 2025
- HGNP: A PCA-based heterogeneous graph neural network for a family distributed flexible job shopComputers & Industrial Engineering, 2025
- A knowledge-driven memetic algorithm for distributed green flexible job shop scheduling considering the endurance of machinesApplied Soft Computing, 2025
- Dynamic production scheduling and maintenance planning under opportunistic groupingComputers & Industrial Engineering, 2024
- Distributed Flexible Job Shop Scheduling through Deploying Fog and Edge Computing in Smart Factories Using Dual Deep Q NetworksMobile Networks and Applications, 2024
- Q-learning-based multi-objective particle swarm optimization with local search within factories for energy-efficient distributed flow-shop scheduling problemJournal of Intelligent Manufacturing, 2023
- A cooperative hierarchical deep reinforcement learning based multi-agent method for distributed job shop scheduling problem with random job arrivalsComputers & Industrial Engineering, 2023
- Dynamic scheduling for flexible job shop using a deep reinforcement learning approachComputers & Industrial Engineering, 2023
- Integrated production and maintenance scheduling for a single degrading machine with deterioration-based failuresComputers & Industrial Engineering, 2020
- Integrated production and preventive maintenance scheduling for a single machine with failure uncertaintyComputers & Industrial Engineering, 2015