A Higher Order Collective Classifier for detecting and classifying network events
Authors:
- 1 January 2009
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 125-130
Abstract
Labeled data is scarce. Most statistical machine learning techniques rely on the availability of a large labeled corpus for building robust models for prediction and classification. In this paper we present a Higher Order Collective Classifier (HOCC) based on higher order learning, a statistical machine learning technique that leverages latent information present in co-occurrences of items across records. These techniques violate the IID assumption that underlies most statistical machine learning techniques and have in prior work outperformed first order techniques in the presence of very limited data. We present results of applying HOCC to two different network data sets, first for detection and classification of anomalies in a border gateway protocol dataset and second for building models of users from network file system calls to perform masquerade detection. The precision of our system has been shown to be 30% better than the standard Naive Bayes technique for masquerade detection. These results indicate that HOCC can successfully model a variety of network events and can be applied to solve difficult problems in security using the general framework proposed.Keywords
This publication has 12 references indexed in Scilit:
- Graph-based text classification2006
- A framework for understanding Latent Semantic Indexing (LSI) performanceInformation Processing & Management, 2006
- Introduction to the special issue on link miningACM SIGKDD Explorations Newsletter, 2005
- An internet routing forensics framework for discovering rules of abnormal BGP eventsACM SIGCOMM Computer Communication Review, 2005
- Dependency Networks for Relational Data2005
- Learning-based anomaly detection in BGP updates2005
- The spread of the Witty wormIEEE Security & Privacy, 2004
- Detection of invalid routing announcement in the Internet2003
- Level search schemes for information filtering and retrievalInformation Processing & Management, 2001
- Corpus-based stemming using cooccurrence of word variantsACM Transactions on Information Systems, 1998