An Adaptive Scheduling Algorithm Integrating Hierarchical Reinforcement Learning and Semi-Markov Decision Processes
Open Access
- 1 July 2026
- journal article
- research article
- Published by MDPI AG in Applied Sciences
- Vol. 16 (13) , 6570
Abstract
Coordinating multiple unmanned aerial vehicle (UAV) systems under strict energy and temporal constraints remains a complex scheduling problem. Existing reinforcement learning methods typically rely on fixed-time-step modeling, which struggles to accommodate flight actions of varying durations and often leads to temporal mismatches between task planning and physical execution. To address this limitation, we propose an Adaptive Hierarchical Semi-Markov Decision Process (AH-SMDP) framework. This architecture decouples task allocation from execution by modeling variable-length actions via an SMDP. An event-driven synchronization mechanism is introduced to align the swarm’s decision-making rhythm with actual task completion times. Additionally, a state-aware reward formulation and a dynamic action space pruning strategy are designed to help UAVs balance energy efficiency with deadline compliance. Simulation results in multi-constraint environments demonstrate that the AH-SMDP framework effectively improves scheduling performance compared to standard MAPPO and PPO algorithms. Under the evaluated experimental settings, the proposed method yields improvements of approximately 30% in average task completion rate, 40% in energy reduction, and 60% in convergence stability. Ablation studies further suggest that this integrated framework offers a viable and effective approach for multi-UAV scheduling.Keywords
Funding Information
- Key Research and Development Program of Henan Province (251111210400)
This publication has 27 references indexed in Scilit:
- Optimizing UAV Task Allocation with Enhanced Battery Efficiency Using Semi-Markov Decision ProcessesJournal of Intelligent & Robotic Systems, 2025
- Three-Dimensional Mobility Management of Unmanned Aerial Vehicles in Flying Ad-Hoc NetworksIEEE Access, 2024
- OHDRL-Based Energy Consumption Optimization for Joint Content Fetching and Trajectory Design of UAVs2024
- A novel energy-efficiency framework for UAV-assisted networks using adaptive deep reinforcement learningScientific Reports, 2024
- Tactical conflict resolution in urban airspace for unmanned aerial vehicles operations using attention-based deep reinforcement learningGreen Energy and Intelligent Transportation, 2023
- Conditional probability based multi-objective cooperative task assignment for heterogeneous UAVsEngineering Applications of Artificial Intelligence, 2023
- Resource-efficient seamless transitions for high-performance multi-hop UAV multicastingComputer Networks, 2022
- Optimal Frequency Reuse and Power Control in Multi-UAV Wireless Networks: Hierarchical Multi-Agent Reinforcement Learning PerspectiveIEEE Access, 2022
- Decentralized task allocation for heterogeneous multi-UAV system with task coupling constraintsThe Journal of Supercomputing, 2020
- Between MDPs and semi-MDPs: A framework for temporal abstraction in reinforcement learningArtificial Intelligence, 1999