Yue-Cai Huang

To improve the network scalability, a large elastic optical network is typically segmented into multiple autonomous domains, where each domain possesses high autonomy and privacy. This architecture is referred to as the multi-domain elastic optical network (MDEON). In the MDEON, the routing, modulation, and spectrum allocation (RMSA) for the inter-domain service requests are challenging. As a result, deep reinforcement learning (DRL) has been introduced recently where the RMSA policies are learned during the interaction of the DRL agents with the MDEON environment. Due to the autonomy of each domain in the MDEON, joint RMSA is essential to improve the overall performance. To realize the joint RMSA, we propose a hierarchical reinforcement learning (HRL) framework which consists of a high-level DRL module and multiple low-level DRL modules (one for each domain), with the collaboration of DRL modules. More specifically, for the inter-domain service requests, the high-level module obtains some abstracted information from the low-level DRL modules and generates the inter-domain RMSA decision for the low-level modules. Then the low-level DRL module gives the intra-domain RMSA decision and feeds back to the high-level module. The proposed HRL framework preserves the autonomy of each single domain while delivering effective overall network performance through the cooperation of the high-level and low-level DRL modules. Simulation results demonstrate that our proposed method outperforms previous approaches.

The reinforcement learning-based routing, modulation, and spectrum assignment has been regarded as an emerging paradigm for resource allocation in the elastic optical networks. One limitation is that the learning process is highly dependent on the training environment, such as the traffic pattern or the optical network topology. Therefore, re-training is required in case of network topology or traffic pattern variations, which consumes a great amount of computation power and time. To ease the requirement of re-training, we propose a policy distillation scheme, which distills knowledge from a well-trained teacher model and then transfers the knowledge to the to-be-trained student model, so that the training of the latter can be accelerated. Specifically, the teacher model is trained for one training environment (e.g., the topology and traffic pattern) and the student model is for another training environment. The simulation results indicate that our proposed method can effectively speed up the training process of the student model, and it even leads to a lower blocking probability, compared with the case that the student model is trained without knowledge distillation.

In this letter, we study the deep reinforcement learning (DRL)-based routing, modulation and spectrum assignment problem in the elastic optical networks. We emphasize the importance of proper reward design of the DRL framework and we propose to include some heuristic information to the reward design. This introduction of human knowledge to the machine learning, is to reduce the exploration blindness of the latter and lead to more efficient learning of better policies. We make it clear that what kind of heuristic information should be included in the reward design is an open question. Specifically, we propose to consider the spectrum fragmentation level of each candidate path as the heuristic information. As a result, the DRL agent is more inclined to choose the candidate path that leads to lower spectrum fragmentation level, which is more friendly for future traffic requests. Simulation results show that the proposed heuristic reward design scheme outperforms both the simple-reward DRL based approaches and the heuristic rule-based approaches.

Deep reinforcement learning (DRL) has been introduced to the routing and spectrum assignment (RSA) of elastic optical networks (EONs) where the RSA policies are learnt during the interaction of a DRL agent with the EON environment. Upon each new traffic arrival, the DRL agent senses the EON state, and makes the RSA decision accordingly. Therefore, the EON state feature extraction is essential for the performance. However, current approaches, mainly based on fully connected neural networks or convolutional neural networks, are unsatisfactory in EON state feature extraction due to the following two reasons. (1) The optical network topology information is not well considered. (2) The path-level features are highly related to the RSA decision, due to the spectrum continuity constraint, while path-level feature extraction has not been well exploited. To overcome the above shortcomings, in this paper, we propose to utilize the Graph Convolutional Neural Network (GCN) and the Recurrent Neural Network (RNN) for the feature extraction of the network topology, and the aggregation of link-level features to path-level features. By doing this, critical RSA-related information can be sensed by the DRL agent to make better actions. Simulation results demonstrate that our proposed method outperforms previous approaches.

Aspect-level sentiment analysis aims to predict the sentiment polarity toward a specific aspect in a sentence. Most current approaches are based on deep learning and the attention mechanism. However, these models cannot simultaneously include the context semantic information carried by local words and the global syntactic information possibly carried by remote words. In this paper, we propose a local semantic and global syntactic integration scheme, which employs a local focus mechanism over local context words and exploits improved graph convolutional networks over dependency tree to encode global syntactic information. Moreover, multi-head attention is used to capture both the semantic information and also the interactive information between semantics and syntactic features. Experimental results on five datasets show the effectiveness of our model over a series of latest models.

We propose a knowledge distillation scheme for deep reinforcement learning-based optical networks. Distilling knowledge from the well-trained model of one traffic pattern to others traffic patterns, so that the latter's training more time-efficient and performance-effective.

A torus-topology photonic data center network with smart flow control is presented, where optical packet switching (OPS), optical circuit switching (OCS), and virtual OCS (VOCS) schemes are all enabled on a unified hardware platform via the combination of hybrid optoelectronic routers (HOPR) and a centralized network controller. An upgraded HOPR is being developed with the goal of handling 100 Gbps optical packets with a high energy efficiency of 90 mW/Gbps and low latency of less than 100 ns. The architecture of HOPR and its enabling technologies are reviewed, including a label processor, optical switch, and optoelectronic shared buffer. We also explain the concept and operation mechanisms of both the data and control planes in the hybrid OPS/OCS/VOCS torus network. The performance of these three transmission modes is evaluated by numerical simulations for a network composed of 4096 HOPRs.