Diagnosing network-wide traffic anomalies
Authors:
Top Cited Papers
Open Access
- 30 August 2004
- journal article
- Published by Association for Computing Machinery (ACM) in ACM SIGCOMM Computer Communication Review
- Vol. 34 (4) , 219-230
Abstract
Anomalies are unusual and significant changes in a network's traffic levels, which can often span multiple links. Diagnosing anomalies is critical for both network operators and end users. It is a difficult problem because one must extract and interpret anomalous patterns from large amounts of high-dimensional, noisy data.In this paper we propose a general method to diagnose anomalies. This method is based on a separation of the high-dimensional space occupied by a set of network traffic measurements into disjoint subspaces corresponding to normal and anomalous network conditions. We show that this separation can be performed effectively by Principal Component Analysis.Using only simple traffic measurements from links, we study volume anomalies and show that the method can: (1) accurately detect when a volume anomaly is occurring; (2) correctly identify the underlying origin-destination (OD) flow which is the source of the anomaly; and (3) accurately estimate the amount of traffic involved in the anomalous OD flow.We evaluate the method's ability to diagnose ( i.e., detect, identify, and quantify) both existing and synthetically injected volume anomalies in real traffic from two backbone networks. Our method consistently diagnoses the largest volume anomalies, and does so with a very low false alarm rate.Keywords
This publication has 15 references indexed in Scilit:
- Structural analysis of network traffic flows2004
- Combining routing and traffic data for detection of IP forwarding anomalies2004
- An information-theoretic approach to traffic matrix estimation2003
- Sketch-based change detection2003
- Multivariate process monitoring and fault diagnosis by multi-scale PCAComputers & Chemical Engineering, 2002
- A signal analysis of network traffic anomalies2002
- Deriving traffic demands for operational IP networks: methodology and experienceIEEE/ACM Transactions on Networking, 2001
- Schemes for fault identification in communication networksIEEE/ACM Transactions on Networking, 1995
- A Singular Value Decomposition Updating Algorithm for Subspace TrackingSIAM Journal on Matrix Analysis and Applications, 1992
- Control Procedures for Residuals Associated With Principal Component AnalysisTechnometrics, 1979