Bixia Tang

The reinforcement learning-based routing, modulation, and spectrum assignment has been regarded as an emerging paradigm for resource allocation in the elastic optical networks. One limitation is that the learning process is highly dependent on the training environment, such as the traffic pattern or the optical network topology. Therefore, re-training is required in case of network topology or traffic pattern variations, which consumes a great amount of computation power and time. To ease the requirement of re-training, we propose a policy distillation scheme, which distills knowledge from a well-trained teacher model and then transfers the knowledge to the to-be-trained student model, so that the training of the latter can be accelerated. Specifically, the teacher model is trained for one training environment (e.g., the topology and traffic pattern) and the student model is for another training environment. The simulation results indicate that our proposed method can effectively speed up the training process of the student model, and it even leads to a lower blocking probability, compared with the case that the student model is trained without knowledge distillation.

In this letter, we study the deep reinforcement learning (DRL)-based routing, modulation and spectrum assignment problem in the elastic optical networks. We emphasize the importance of proper reward design of the DRL framework and we propose to include some heuristic information to the reward design. This introduction of human knowledge to the machine learning, is to reduce the exploration blindness of the latter and lead to more efficient learning of better policies. We make it clear that what kind of heuristic information should be included in the reward design is an open question. Specifically, we propose to consider the spectrum fragmentation level of each candidate path as the heuristic information. As a result, the DRL agent is more inclined to choose the candidate path that leads to lower spectrum fragmentation level, which is more friendly for future traffic requests. Simulation results show that the proposed heuristic reward design scheme outperforms both the simple-reward DRL based approaches and the heuristic rule-based approaches.