Searching for a Needle in a Haystack: Predicting Security Vulnerabilities for Windows Vista
Authors:
- Thomas ZimmermannTZThomas Zimmermann
- Nachiappan NagappanNNNachiappan Nagappan
- Laurie WilliamsLWLaurie Williams
Top Cited Papers
- 1 January 2010
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- No. 21594848,p. 421-428
Abstract
Many factors are believed to increase the vulnerability of software system; for example, the more widely deployed or popular is a software system the more likely it is to be attacked. Early identification of defects has been a widely investigated topic in software engineering research. Early identification of software vulnerabilities can help mitigate these attacks to a large degree by focusing better security verification efforts in these components. Predicting vulnerabilities is complicated by the fact that vulnerabilities are, most often, few in number and introduce significant bias by creating a sparse dataset in the population. As a result, vulnerability prediction can be thought of us preverbally “searching for a needle in a haystack.” In this paper, we present a large-scale empirical study on Windows Vista, where we empirically evaluate the efficacy of classical metrics like complexity, churn, coverage, dependency measures, and organizational structure of the company to predict vulnerabilities and assess how well these software measures correlate with vulnerabilities. We observed in our experiments that classical software measures predict vulnerabilities with a high precision but low recall values. The actual dependencies, however, predict vulnerabilities with a lower precision but substantially higher recall.Keywords
This publication has 16 references indexed in Scilit:
- Toward Non-security Failures as a Predictor of Security Faults and Failures2009
- The Evolution and Decay of Statically Detected Source Code Vulnerabilities2008
- The influence of organizational structure on software quality2008
- Have things changed now?2006
- Predicting component failures at design time2006
- Mining metrics to predict component failures2006
- Building knowledge through families of experimentsIEEE Transactions on Software Engineering, 1999
- A training algorithm for optimal margin classifiers1992
- The detection of fault-prone programsIEEE Transactions on Software Engineering, 1992
- A Complexity MeasureIEEE Transactions on Software Engineering, 1976