LARGen: Automatic Signature Generation for Malwares Using Latent Dirichlet Allocation
- 15 September 2016
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Dependable and Secure Computing
- Vol. 15 (5) , 771-783
Abstract
As the quantity and complexity of network threats grow, Intrusion Detection Systems (IDSs) have become critical for securing networks. Achieving computer network intrusion detection with these IDSs requires high-level information technology and security expertise because malicious traffic has to be rigorously analyzed and the appropriate IDS rules written to effectively detect vulnerabilities that may potentially be exploited. However, incorrect IDS rules may produce numerous false positives, thereby degrading the performance of the IDS, and even worse, paralyzing the network. In this paper, we present a novel approach that exploits the Latent Dirichle Allocation (LDA) algorithm to generate IDS rules. Our proposed method, called LDA-based Automatic Rule Generation (LARGen), automatically performs an analysis of the malicious traffic and extracts the appropriate attack signatures that will be used for IDS rules. LARGen first extracts multiple signature strings embedded in network flows. Then, the flows are classified based on the extracted signature strings, and key content strings for malicious traffic are identified through the LDA inferential topic model. Those key content strings are the core of an IDS rule that can detect malicious traffic. We study the effectiveness of LDA in the context of network attack signature generation via extensive experiments with real network trace data, consisting of both benign and malicious traffic. Experimental results confirm that threat rules generated from LARGen accurately detect every cyber attack with high accuracy.Keywords
Funding Information
- Korea National University of Transportation
- Basic Science Research Program of the National Research Foundation of Korea (NRF-2014R1A1A2053456)
This publication has 39 references indexed in Scilit:
- Statistical topic models for multi-label document classificationMachine Learning, 2011
- NeTraMarkACM SIGCOMM Computer Communication Review, 2011
- NetShieldACM SIGCOMM Computer Communication Review, 2010
- Auto-Sign: an automatic signature generator for high-speed malware filtering devicesJournal of Computer Virology and Hacking Techniques, 2009
- ShieldACM SIGCOMM Computer Communication Review, 2004
- Finding scientific topicsProceedings of the National Academy of Sciences of the United States of America, 2004
- The expectation-maximization algorithmIEEE Signal Processing Magazine, 1996
- A parameterizable methodology for Internet traffic flow profilingIEEE Journal on Selected Areas in Communications, 1995
- On Gibbs sampling for state space modelsBiometrika, 1994
- Multiple Hypergeometric Functions: Probabilistic Interpretations and Statistical UsesJournal of the American Statistical Association, 1983