EGAD: Evolving Graph Representation Learning with Self-Attention and Knowledge Distillation for Live Video Streaming Events
Preprint
Abstract
In this study, we present a dynamic graph representation learning model on weighted graphs to accurately predict the network capacity of connections between viewers in a live video streaming event. We propose EGAD, a neural network architecture to capture the graph evolution by introducing a self-attention mechanism on the weights between consecutive graph convolutional networks. In addition, we account for the fact that neural architectures require a huge amount of parameters to train, thus increasing the online inference latency and negatively influencing the user experience in a live video streaming event. To address the problem of the high online inference of a vast number of parameters, we propose a knowledge distillation strategy. In particular, we design a distillation loss function, aiming to first pretrain a teacher model on offline data, and then transfer the knowledge from the teacher to a smaller student model with less parameters. We evaluate our proposed model on the link prediction task on three real-world datasets, generated by live video streaming events. The events lasted 80 minutes and each viewer exploited the distribution solution provided by the company Hive Streaming AB. The experiments demonstrate the effectiveness of the proposed model in terms of link prediction accuracy and number of required parameters, when evaluated against state-of-the-art approaches. In addition, we study the distillation performance of the proposed model in terms of compression ratio for different distillation strategies, where we show that the proposed model can achieve a compression ratio up to 15:100, preserving high link prediction accuracy. For reproduction purposes, our evaluation datasets and implementation are publicly available at https://stefanosantaris.github.io/EGAD.Keywords
All Related Versions
This publication has 15 references indexed in Scilit:
- Deep Geometric Knowledge Distillation with Graphs2020
- Dynamic Joint Variational Graph Autoencoders2020
- dyngraph2vec: Capturing network dynamics using dynamic graph representation learningKnowledge-Based Systems, 2020
- Knowledge Distillation via Instance Relationship Graph2019
- DeepVID: Deep Visual Interpretation and Diagnosis for Image Classifiers via Knowledge DistillationIEEE Transactions on Visualization and Computer Graphics, 2019
- Destination-aware Adaptive Traffic Flow Rule Aggregation in Software-Defined Networks2019
- Structured Sequence Modeling with Graph Convolutional Recurrent Networks2018
- Video delivery networks: Challenges, solutions and future directionsComputers and Electrical Engineering, 2018
- Multimedia streaming in information-centric networking: A survey and future perspectivesComputer Networks, 2017
- A Gift from Knowledge Distillation: Fast Optimization, Network Minimization and Transfer Learning2017