Maria Grazia Marchesano

Effective production control requires aligning strategic planning with real-time execution under dynamic and stochastic conditions. This study proposes a scalable dual-agent Deep Reinforcement Learning (DRL) framework for the joint optimisation of Work-In-Process (WIP) control and job sequencing in flow-shop environments. A strategic DQN agent regulates global WIP to meet throughput targets, while a tactical DQN agent adaptively selects dispatching rules at the machine level on an event-driven basis. Parameter sharing in the tactical agent ensures inherent scalability, overcoming the combinatorial complexity of multi-machine scheduling. The agents coordinate indirectly via a shared simulation environment, learning to balance global stability with local responsiveness. The framework is validated through a discrete-event simulation integrating agent-based modelling, demonstrating consistent performance across multiple production scales (5–15 machines) and process time variabilities. Results show that the approach matches or surpasses analytical benchmarks and outperforms static rule-based strategies, highlighting its robustness, adaptability, and potential as a foundation for future Hierarchical Reinforcement Learning applications in manufacturing.

This study addresses the scheduling problem in a single-machine system with variable processing speed. The performance of the Earliest Due Date (EDD) rule, operating at a fixed maximum speed, is compared with a Reinforcement Learning (RL)-based approach capable of dynamically adapting both processing speed and job selection. The analysis considers both temporal performance (tardiness) and sustainability (energy consumption). Simulation results, supported by statistical analysis, show that the RL approach achieves a significant reduction in total energy consumption compared to the EDD rule. Regarding temporal performance, analysis of variance reveals that the chosen policy significantly affects the number of tardy jobs, with an effect that depends on the tightness of job deadlines, whereas the overall average tardiness across all jobs does not differ significantly between the two policies. These findings highlight the potential of adaptive RL-based policies for more sustainable resource management, demonstrating the ability to achieve substantial energy savings while maintaining competitive temporal performance and dynamically managing the trade-off according to deadline pressure.

Manufacturing organisations face increasing pressure to assess and report their sustainability performance, particularly with the introduction of the European Sustainability Reporting Standards (ESRS). This paper presents a novel sustainability assessment framework that integrates ESRS guidelines with the Analytic Hierarchy Process (AHP) to provide a comprehensive evaluation methodology for manufacturing organisations. The framework addresses the limitations of existing approaches by systematically incorporating standardised sustainability indicators while enabling precise weighting of their relative importance. Through a structured approach, the framework evaluates environmental, social, and governance dimensions using both measurable and non-measurable indicators derived from ESRS guidelines. The methodology includes automated pairwise comparisons and weight calculations, significantly reducing the complexity traditionally associated with multi-criteria decision-making methods. The framework’s effectiveness is demonstrated through its implementation in a manufacturing organisation, where it successfully evaluated 38 sustainability indicators across 10 ESRS-defined subtopics. The case study highlighted areas requiring immediate attention, particularly in water and marine resources management, where performance fell below the established benchmark. This research contributes to both theory and practice by offering manufacturing organisations a structured approach to sustainability assessment that combines regulatory compliance with effective performance evaluation. The framework’s automated nature and integration of standardised reporting requirements make it particularly valuable for organisations transitioning to mandatory sustainability reporting under ESRS guidelines.

This study explores advanced process monitoring for one-shot drilling of aeronautical stacks made of aluminium 2024 and carbon fibre-reinforced polymer (CFRP) laminates using a 4.8 mm diameter drilling tool and unsupervised machine learning techniques. An experimental campaign is conducted to collect thrust force and torque signals at a 10 kHz sampling rate during the drilling process. These signals are employed for real-time process monitoring, focusing on material change detection and anomaly identification, where anomalies are defined as holes that fail to meet predefined quality criteria. An innovative approach based on unsupervised learning is proposed to enable automatic material change identification, signal segmentation, feature extraction, and hole quality assessment. Specifically, a semi-supervised approach based on a Gaussian Mixture Model (GMM) and 1D Convolutional AutoEncoder (1D-CAE) is employed to detect deviations from normal drilling conditions. The proposed method is benchmarked against state-of-the-art supervised techniques, including logistic regression (LR) and Support Vector Machines (SVMs). Results show that these traditional models struggle with class imbalance, leading to overfitting and limited generalisation, as reflected by the F1 scores of 0.78 and 0.75 for LR and SVM, respectively. In contrast, the proposed semi-supervised approach improves anomaly detection, achieving an F1 score of 0.87 by more effectively identifying poor-quality holes. This study demonstrates the potential of deep learning-based semi-supervised methods for intelligent process monitoring, enabling adaptive control in the drilling process of hybrid stacks and detecting anomalous holes. While the proposed approach effectively handles small and imbalanced datasets, further research into the application of generative AI could enhance performance, aiming for F1 scores above 0.90, thereby supporting adaptation in real industrial environments with high performance.

This study presents an optimisation framework for operating a battery swapping station (BSS) to enhance efficiency and sustainability in electric vehicle (EV) infrastructure. A hybrid modelling approach combines agent-based discrete event simulation and linear programming to model the dynamic behaviour of batteries and operational processes within the BSS. The model considers factors such as charging speed, battery degradation, grid power constraints, customer behaviour, and range anxiety. The agent-based model simulates the interaction between vehicles, batteries, and the station, capturing the stochastic nature of EVs’ arrivals and battery demand with the discrete event simulation. The linear programming component optimises battery state transitions to minimise degradation and ensure that the demand is met while respecting the power limits of the grid. Different battery types are considered based on vehicle category, each with specific capacity and usage patterns, reflecting real-world market conditions. The results demonstrate that the proposed optimisation framework can effectively manage the complex operational needs of a BSS. The proposed framework effectively balances service quality with resource efficiency by employing a strategic mix of charging modes and inventory management, reducing operational and degradation costs. This approach supports a more sustainable EV infrastructure, highlighting BSS as a viable solution to enhance the efficiency and sustainability of EV operations. Furthermore, the analysis highlights the critical role of power limits in determining charging strategies and their impact on operational efficiency. The findings suggest that with optimised operations, BSS can play a critical role in accelerating the adoption of EVs by offering a faster, more reliable, and sustainable alternative.

The integration of Battery Swapping Stations (BSSs) into smart microgrids presents an opportunity to optimize energy generation, storage, and consumption. However, there exists a gap in the literature regarding the detailed analysis of the profitability of integrating a BSS within a smart microgrid, particularly utilizing second-life batteries for storage and renewable energy sources for generation. This study aims to address this gap by employing a multi-method simulation approach to thoroughly investigate the economic viability of such integration. The simulation model developed for this study is a digital twin of the microgrid, incorporating components such as the BSS, renewable energy sources (wind and photovoltaic), second-life battery storage, and utilities. By optimizing energy flows within this model, considering the cost-effectiveness of diverse generation sources and prioritizing the utilization of renewable energy, we aim to provide a comprehensive assessment of the economic benefits. Furthermore, the simulation takes into account crucial factors including battery swapping operations, warehouse management, and battery charging scheduling. The profitability analysis undertaken in this study is grounded in the objective of minimizing total costs while effectively meeting the energy demands of residential loads. Ultimately, the integration of the BSS into the smart microgrid not only targets economic efficiency but also strives to maximize the utilization of second-life batteries, contributing to the concept of a circular economy.