Explainable Multi-Agent Reinforcement Learning for Extended Reality Codec Adaptation
Open Access
- 18 December 2025
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Machine Learning in Communications and Networking
- Vol. 4, 245-264
Abstract
Extended Reality (XR) services are set to transform applications over 5th and 6th generation wireless networks, delivering immersive experiences. Concurrently, Artificial Intelligence (AI) advancements have expanded their role in wireless networks, however, trust and transparency in AI remain to be strengthened. Thus, providing explanations for AI-enabled systems can enhance trust. We introduce Value Function Factorization (VFF)-based Explainable (X) Multi-Agent Reinforcement Learning (MARL) algorithms, explaining reward design in XR codec adaptation through reward decomposition.We contribute four enhancements to XMARL algorithms. Firstly, we detail architectural modifications to enable reward decomposition in VFF-based MARL algorithms: Value Decomposition Networks (VDN), Mixture of Q-Values (QMIX), and Q-Transformation (Q-TRAN). Secondly, inspired by multi-task learning, we reduce the overhead of vanilla XMARL algorithms. Thirdly, we propose a new explainability metric, Reward Difference Fluctuation Explanation (RDFX), suitable for problems with adjustable parameters. Lastly, we propose adaptive XMARL, leveraging network gradients and reward decomposition for improved action selection. Simulation results indicate that, in XR codec adaptation, the Packet Delivery Ratio reward is the primary contributor to optimal performance compared to the initial composite reward, which included delay and Data Rate Ratio components. Modifications to VFF-based XMARL algorithms, incorporating multi-headed structures and adaptive loss functions, enable the best-performing algorithm, Multi-Headed Adaptive (MHA)-QMIX, to achieve significant average gains over the Adjust Packet Size baseline up to 10.7%, 41.4%, 33.3%, and 67.9% in XR index, jitter, delay, and Packet Loss Ratio (PLR), respectively.Keywords
All Related Versions
Funding Information
- Natural Sciences and Engineering Research Council of Canada (NSERC) Canada Research Chairs Program
- Mitacs Accelerate Program
- Telefonaktiebolaget LM Ericsson
This publication has 29 references indexed in Scilit:
- Enhancing 5G QoS Management for XR Traffic Through XR Loopback MechanismIEEE Journal on Selected Areas in Communications, 2023
- 6G: The Next Giant Leap for AI and MLProcedia Computer Science, 2023
- Toward native explainable and robust AI in 6G networks: Current state, challenges and road aheadComputer Communications, 2022
- Proposed QoE Models Associated with Delay and Jitter Using Subjective Approach and Applications for 4G and 5G Networks2021
- Explainable Deep Reinforcement Learning for UAV autonomous path planningAerospace Science and Technology, 2021
- A Comprehensive Survey of the Tactile Internet: State-of-the-Art and Research DirectionsIEEE Communications Surveys & Tutorials, 2020
- Peeking Inside the Black-Box: A Survey on Explainable Artificial Intelligence (XAI)IEEE Access, 2018
- A Concise Introduction to Decentralized POMDPs2016
- Multitask LearningMachine Learning, 1997
- Markov games as a framework for multi-agent reinforcement learning1994