A novel semi-supervised approach for network traffic clustering
- 1 September 2011
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 169-175
Abstract
Network traffic classification is an essential component for network management and security systems. To address the limitations of traditional port-based and payload-based methods, recent studies have been focusing on alternative approaches. One promising direction is applying machine learning techniques to classify traffic flows based on packet and flow level statistics. In particular, previous papers have illustrated that clustering can achieve high accuracy and discover unknown application classes. In this work, we present a novel semi-supervised learning method using constrained clustering algorithms. The motivation is that in network domain a lot of background information is available in addition to the data instances themselves. For example, we might know that flow f1 and f2 are using the same application protocol because they are visiting the same host address at the same port simultaneously. In this case, f1 and f2 shall be grouped into the same cluster ideally. Therefore, we describe these correlations in the form of pair-wise must-link constraints and incorporate them in the process of clustering. We have applied three constrained variants of the K-Means algorithm, which perform hard or soft constraint satisfaction and metric learning from constraints. A number of real-world traffic traces have been used to show the availability of constraints and to test the proposed approach. The experimental results indicate that by incorporating constraints in the course of clustering, the overall accuracy and cluster purity can be significantly improved.Keywords
This publication has 16 references indexed in Scilit:
- Digging into HTTPS2010
- Challenging statistical classification for operational usage2009
- A survey of techniques for internet traffic classification using machine learningIEEE Communications Surveys & Tutorials, 2008
- K-Means with Large and Noisy Constraint Sets2007
- Semi-supervised network traffic classificationACM SIGMETRICS Performance Evaluation Review, 2007
- Bayesian Neural Networks for Internet Traffic ClassificationIEEE Transactions on Neural Networks, 2007
- Unexpected means of protocol inference2006
- Traffic classification using clustering algorithms2006
- Traffic classification on the flyACM SIGCOMM Computer Communication Review, 2006
- Integrating constraints and metric learning in semi-supervised clustering2004