Abstract
Open Radio Access Network (O-RAN) introduces xApps to enable flexible, near real-time control of RAN functions. However, managing multiple xApps dynamically, under rapidly changing network conditions, becomes significantly more challenging in IoT-driven environments, where massive numbers of low-power devices may simultaneously access the network, triggering phenomena such as wake-up storms. In this work, we showcase and compare Digital Twin-enhanced Multi-Agent Reinforcement Learning (MARL) strategies for intelligent xApp management in IoT-enabled O-RAN systems. We explore how Digital Twins can simulate large-scale, event-driven IoT device behaviors, enabling proactive optimization of xApp scheduling while reducing communication and synchronization overhead. We further examine advanced MARL approaches, such as Hierarchical Agent Policy Optimization (HAPPO), to address scalability and coordination challenges under ultra-dense, low-latency, and energy-constrained LPWAN protocols like NB-IoT and LTE-M. We define two system-level KPIs to evaluate the performance of RL schemes: Conflict Avoidance Efficiency and Coordination QoS Improvement. The proposed MARL-based approach significantly outperforms conventional single-agent RL methods in both coordination effectiveness and overall QoS delivery, while maintaining a reasonable model complexity suitable for practical deployment.
Funding Information
  • CHEDDAR: Communications Hub for Empowering Distributed Cloud Computing Applications and Research funded by the UK EPSRC (EP/Y037421/1)

This publication has 11 references indexed in Scilit: