Darknet and deepnet mining for proactive cybersecurity threat intelligence
Authors:
- 1 September 2016
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 7-12
Abstract
In this paper, we present an operational system for cyber threat intelligence gathering from various social platforms on the Internet particularly sites on the darknet and deepnet. We focus our attention to collecting information from hacker forum discussions and marketplaces offering products and services focusing on malicious hacking. We have developed an operational system for obtaining information from these sites for the purposes of identifying emerging cyber threats. Currently, this system collects on average 305 high-quality cyber threat warnings each week. These threat warnings include information on newly developed malware and exploits that have not yet been deployed in a cyber-attack. This provides a significant service to cyber-defenders. The system is significantly augmented through the use of various data mining and machine learning techniques. With the use of machine learning models, we are able to recall 92% of products in marketplaces and 80% of discussions on forums relating to malicious hacking with high precision. We perform preliminary analysis on the data collected, demonstrating its application to aid a security expert for better threat analysis.Keywords
All Related Versions
This publication has 14 references indexed in Scilit:
- It’s Dark in There: Using Systems Analysis to Investigate Trust and Engagement in Dark Web Forums2015
- An analysis of underground forums2011
- A focused crawler for Dark Web forumsJournal of the American Society for Information Science and Technology, 2010
- Exploring stolen data markets online: products and market forcesCriminal Justice Studies, 2010
- subcultural evolution? examining the influence of on- and off-line experiences on deviant subculturesDeviant Behavior, 2007
- A Closed Form Solution to Natural Image Matting2006
- Topical web crawlersACM Transactions on Internet Technology, 2004
- Focused crawling: a new approach to topic-specific Web resource discoveryComputer Networks, 1999
- A Sociology of HackersThe Sociological Review, 1998
- Combining labeled and unlabeled data with co-training1998