Prohim Tam

The last decade has witnessed the explosive growth of the internet of things (IoT), demonstrating the utilization of ubiquitous sensing and computation services. Hence, the industrial IoT (IIoT) is integrated into IoT devices. IIoT is concerned with the limitation of computation and battery life. Therefore, mobile edge computing (MEC) is a paradigm that enables the proliferation of resource computing and reduces network communication latency to realize the IIoT perspective. Furthermore, an open radio access network (O-RAN) is a new architecture that adopts a MEC server to offer a provisioning framework to address energy efficiency and reduce the congestion window of IIoT. However, dynamic resource computation and continuity of task generation by IIoT lead to challenges in management and orchestration (MANO) and energy efficiency. In this article, we aim to investigate the dynamic and priority of resource management on demand. Additionally, to minimize the long-term average delay and computation resource-intensive tasks, the Markov decision problem (MDP) is conducted to solve this problem. Hence, deep reinforcement learning (DRL) is conducted to address the optimal handling policy for MEC-enabled O-RAN architectures. In this study, MDP-assisted deep q-network-based priority/demanding resource management, namely DQG-PD, has been investigated in optimizing resource management. The DQG-PD algorithm aims to solve resource management and energy efficiency in IIoT devices, which demonstrates that exploiting the deep Q-network (DQN) jointly optimizes computation and resource utilization of energy for each service request. Hence, DQN is divided into online and target networks to better adapt to a dynamic IIoT environment. Finally, our experiment shows that our work can outperform reference schemes in terms of resources, cost, energy, reliability, and average service completion ratio.

AI-empowered applications have been deployed in many aspects of networking, and federated learning (FL) has emerged as a complementary approach due to its ability to enable privacy-preserving model training and inference. However, without self-organizing capability, practical FL systems face several issues to co-exist in real-time networking. Therefore, this paper aims to design autonomous FL management with integrated graph neural networks (GNN) and deep reinforcement learning (DRL), namely AutoFedGDRL, to sustain heterogeneous FL execution in optimized open radio access network (O-RAN) and intelligent core network architectures and offer automated policy-driven orchestration by intelligent agent controller. Edge cloud virtualized O-RAN is integrated to assist model computation and support multiple services with elastic containerized resource scaling. The practicability of FL systems is stimulated by modelling the participants and aggregators as a graph representation and subsequently analyzing to predict the accessibility and trustworthiness of the nodes, bandwidth capacities, and virtual link relationship. Our proposed AutoFedGDRL aims to obtain specifications of hidden FL, service, and networking states in order to control the main policies, such as training management, resource sharing, aggregation scheduling, and service prioritization. In the experiment, AutoFedGDRL surpassed reference models (non-federated training) in global accuracy, achieving 98.23% for MNIST and 97.12% for CIFAR-10, compared to 98.22% and 95.89% for PrimaryGNN-FL. The proposed scheme also improved end-to-end convergence speed, with execution times 10.58 ms to 32.79 ms faster. Model delivery ratios reached 99.98%, ensuring the federated system's reliability and sharing workload efficiency.

Network functions virtualization (NFV) has become the platform for decomposing the sequence of virtual network functions (VNFs), which can be grouped as a forwarding graph of service function chaining (SFC) to serve multi-service slice requirements. NFV-enabled SFC consists of several challenges in reaching the reliability and efficiency of key performance indicators (KPIs) in management and orchestration (MANO) decision-making control. The problem of SFC fault tolerance is one of the most critical challenges for provisioning service requests, and it needs resource availability. In this article, we proposed graph neural network (GNN)-based deep reinforcement learning (DRL) to enhance SFC fault tolerance (GRL-SFT), which targets the chain graph representation, long-term approximation, and self-organizing service orchestration for future massive Internet of Everything applications. We formulate the problem as the Markov decision process (MDP). DRL seeks to maximize the cumulative rewards by maximizing the service request acceptance ratios and minimizing the average completion delays. The proposed model solves the VNF management problem in a short time and configures the node allocation reliably for real-time restoration. Our simulation result demonstrates the effectiveness of the proposed scheme and indicates better performance in terms of total rewards, delays, acceptances, failures, and restoration ratios in different network topologies compared to reference schemes.

Internet of Things (IoT) devices can integrate with applications requiring intensive contextual data processing, intelligent vehicle control, healthcare remote sensing, VR, data mining, traffic management, and interactive applications. However, there are computationally intensive tasks that need to be completed quickly within the time constraints of IoT devices. To address this challenge, researchers have proposed computation offloading, where computing tasks are sent to edge servers instead of being executed locally on user devices. This approach involves using edge servers located near users in cellular network base stations, and also known as Mobile Edge Computing (MEC). The goal is to offload tasks to edge servers, optimizing both latency and energy consumption. The main objective of this paper mentioned in the summary is to design an algorithm for time- and energy-optimized task offloading decision-making in MEC environments. Therefore, we developed a Lagrange Duality Resource Optimization Algorithm (LDROA) to optimize for both decision offloading and resource allocation for tasks, whether to locally execute or offload to an edge server. The LDROA technique produces superior simulation outcomes in terms of task offloading, with improved performance in computation latency and cost usage compared to conventional methods like Random Offloading, Load Balancing, and the Greedy Latency Offloading scheme.

This paper provides a comprehensive survey of the integration of graph neural networks (GNN) and deep reinforcement learning (DRL) in end-to-end (E2E) networking solutions. We delve into the fundamentals of GNN, its variants, and the state-of-the-art applications in communication networking, which reveal the potential to revolutionize access, transport, and core network management policies. This paper further explores DRL capabilities, its variants, and the trending applications in E2E networking, particularly in enhancing dynamic network (re)configurations and resource management. By fusing GNN with DRL, we spotlight novel approaches, ranging from radio access networks to core management and orchestration, across E2E network layers. Deployment scenarios in smart transportation, smart factory, and smart grids demonstrate the practical implications of our survey topic. Lastly, we point out potential challenges and future research directions, including the critical aspects for modelling explainability, the reduction in overhead consumption, interoperability with existing schemes, and the importance of reproducibility. Our survey aims to serve as a roadmap for future developments in E2E networking, guiding through the current landscape, challenges, and prospective breakthroughs in the algorithm modelling toward network automation using GNN and DRL.

Network slicing is introduced for elastically instantiating logical network infrastructure isolation to support different application types with diversified quality of service (QoS) class indicators. In particular, vehicular communications are a trending area that consists of massive mission-critical applications in the range of safety-critical, intelligent transport systems, and on-board infotainment. Slicing management can be achieved if the network infrastructure has computing sufficiency, a dynamic control policy, elastic resource virtualization, and cross-tier orchestration. To support the functionality of slicing management, incorporating core network infrastructure with deep learning and reinforcement learning has become a hot topic for researchers and practitioners in analyzing vehicular traffic/resource patterns before orchestrating the steering policies. In this paper, we propose QoS-driven management by considering (edge) resource block utilization, scheduling, and slice instantiation in a three-tier resource placement, namely, small base stations/access points, macro base stations, and core networks. The proposed scheme integrates recurrent neural networks to trigger hidden states of resource availability and predict the output of QoS. The intelligent agent and slice controller, namely, RDQ3N, gathers the resource states from three-tier observations and optimizes the action on allocation and scheduling algorithms. Experiments are conducted on both physical and virtual representational vehicle-to-everything (V2X) environments; furthermore, service requests are set to massive thresholds for rendering V2X congestion flow entries.

In the core networking of smart cities, mobile network operators need solutions to reflect service function chaining (SFC) orchestration policies while ensuring efficient resource utilization and preserving quality of service (QoS) in large-scale networking congestion states. To offer this solution, we observe the standardized QoS class identifiers of smart city scenarios. Then, we reflect the service criticalities via cloning virtual network function (VNF) with reserved resources for ensuring effective scheduling of request queue management. We employ graph neural networks (GNN) with a message-passing mechanism to iteratively update hidden states of VNF nodes with the objectives of enhancing allocation of resource blocks, accurate detection of availability statuses, and duplication of heavily congested instances. The deployment properties of smart city use cases are presented along with their intelligent service functions, and we aim to activate a modular architecture with multi-purpose VNFs and chaining isolation for generalizing global instances. Experimental simulation is conducted to illustrate how the proposed scheme performs under different congestion levels of SFC request rates, while capturing the key performance metrics of average delay, acceptance ratios, and completion ratios.

The advancement of the sensing capabilities of end devices drives a variety of data-intensive insights, yielding valuable information for modelling intelligent industrial applications. To apply intelligent models in 5G and beyond, edge intelligence integrates edge computing systems and deep learning solutions, which enables distributed model training and inference. Edge federated learning (EFL) offers collaborative edge intelligence learning with distributed aggregation capabilities, promoting resource efficiency, participant inclusivity, and privacy preservation. However, the quality of service (QoS) faces challenges due to congestion problems that arise from the diverse models and data in practical architectures. In this paper, we develop a modified long short-term memory (LSTM)-based congestion-aware EFL (MLSTM-CEFL) approach that aims to enhance QoS in the final model convergence between end devices, edge aggregators, and the global server. Given the diversity of service types, MLSTM-CEFL proactively detects the congestion rates, adequately schedules the edge aggregations, and effectively prioritizes high mission-critical serving resources. The proposed system is formulated to handle time series analysis from local/edge model parameter loading, weighing the configuration of resource pooling properties at specific congestion intervals. The MLSTM-CEFL policy orchestrates the establishment of long-term paths for participant-aggregator scheduling and follows the expected QoS metrics after final averaging in multiple industrial application classes.

The limited coverage extension of mobile edge computing (MEC) necessitates exploring cooperation with unmanned aerial vehicles (UAV) to leverage advanced features for future computation-intensive and mission-critical applications. Moreover, the workflow for task offloading in software-defined networking (SDN)-enabled 5G is significant to tackle in UAV-MEC networks. In this paper, deep reinforcement learning (DRL) SDN control methods for improving computing resources are proposed. DRL-based SDN controller, termed DRL-SDNC, allocates computational resources, bandwidth, and storage based on task requirements, upper-bound tolerable delays, and network conditions, using the UAV system architecture for task exchange between MECs. DRL-SDNC configures rule installation based on state observations and agent evaluation indicators, such as network congestion, user equipment computational capabilities, and energy efficiency. This paper also proposes the training deep network architecture for the DRL-SDNC, enabling interactive and autonomous policy enforcement. The agent learns from the UAV-MEC environment through experience gathering and updates its parameters using optimization methods. DRL-SDNC collaboratively adjusts hyperparameters and network architecture to enhance learning efficiency. Compared with baseline schemes, simulation results demonstrate the effectiveness of the proposed approach in optimizing resource efficiency and achieving satisfied quality of service for efficient utilization of computing and communication resources in UAV-assisted networking environments.

To build intelligent model learning in conventional architecture, the local data are required to be transmitted toward the cloud server, which causes heavy backhaul congestion, leakage of personalization, and insufficient use of network resources. To address these issues, federated learning (FL) is introduced by offering a systematical framework that converges the distributed modeling process between local participants and the parameter server. However, the challenging issues of insufficient participant scheduling, aggregation policies, model offloading, and resource management still remain within conventional FL architecture. In this survey article, the state-of-the-art solutions for optimizing the orchestration in FL communications are presented, primarily querying the deep reinforcement learning (DRL)-based autonomy approaches. The correlations between the DRL and FL mechanisms are described within the optimized system architectures of selected literature approaches. The observable states, configurable actions, and target rewards are inquired into to illustrate the applicability of DRL-assisted control toward self-organizing FL systems. Various deployment strategies for Internet of Things applications are discussed. Furthermore, this article offers a review of the challenges and future research perspectives for advancing practical performances. Advanced solutions in these aspects will drive the applicability of converged DRL and FL for future autonomous communication-efficient and privacy-aware learning.