Digital Twin-Enhanced Reinforcement Learning for Intelligent xApps Management in O-RAN Systems
- 24 October 2025
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Internet of Things Magazine
- Vol. PP (25763180) , 1-8
Abstract
Open Radio Access Network (O-RAN) introduces xApps to enable flexible, near real-time control of RAN functions. However, managing multiple xApps dynamically, under rapidly changing network conditions, becomes significantly more challenging in IoT-driven environments, where massive numbers of low-power devices may simultaneously access the network, triggering phenomena such as wake-up storms. In this work, we showcase and compare Digital Twin-enhanced Multi-Agent Reinforcement Learning (MARL) strategies for intelligent xApp management in IoT-enabled O-RAN systems. We explore how Digital Twins can simulate large-scale, event-driven IoT device behaviors, enabling proactive optimization of xApp scheduling while reducing communication and synchronization overhead. We further examine advanced MARL approaches, such as Hierarchical Agent Policy Optimization (HAPPO), to address scalability and coordination challenges under ultra-dense, low-latency, and energy-constrained LPWAN protocols like NB-IoT and LTE-M. We define two system-level KPIs to evaluate the performance of RL schemes: Conflict Avoidance Efficiency and Coordination QoS Improvement. The proposed MARL-based approach significantly outperforms conventional single-agent RL methods in both coordination effectiveness and overall QoS delivery, while maintaining a reasonable model complexity suitable for practical deployment.Keywords
Funding Information
- CHEDDAR: Communications Hub for Empowering Distributed Cloud Computing Applications and Research funded by the UK EPSRC (EP/Y037421/1)
This publication has 11 references indexed in Scilit:
- Toward Sustainability in 6G and Beyond: Challenges and Opportunities of Open RANIEEE Communications Standards Magazine, 2025
- Open RAN for 6G Networks: Architecture, Use Cases and Open IssuesIEEE Communications Surveys & Tutorials, 2025
- Contrastive Learning for Distortion Tolerable Network Slice Prediction in Open RAN2025
- Multiple Access via Curriculum Multitask HAPPO Based on Dynamic Heterogeneous Wireless NetworkIEEE Internet of Things Journal, 2024
- ORAN-B5G: A Next-Generation Open Radio Access Network Architecture With Machine Learning for Beyond 5G in Industrial 5.0IEEE Transactions on Green Communications and Networking, 2024
- Emergent Communication in Multi-Agent Reinforcement Learning for Future Wireless NetworksIEEE Internet of Things Magazine, 2023
- Digital Twin for Open RAN: Toward Intelligent and Resilient 6G Radio Access NetworksIEEE Communications Magazine, 2023
- Cooperative UAV Resource Allocation and Task Offloading in Hierarchical Aerial Computing Systems: A MAPPO-Based ApproachIEEE Internet of Things Journal, 2023
- Networked Twins and Twins of Networks: An Overview on the Relationship Between Digital Twins and 6GIEEE Communications Standards Magazine, 2021
- Attn-CommNet: Coordinated Traffic Lights Control On Large-Scale Network Level2021