HEDF: A Method for Early Forecasting Software Defects Based on Human Error Mechanisms
Open Access
- 5 January 2023
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 11, 3626-3652
Abstract
As the primary cause of software defects, human error is the key to understanding, and perhaps to forecasting and avoiding defects. Little research has been done to forecast defects on the basis of the cognitive errors that cause them. The existing “defect prediction” models are applied to code once it has been produced: therefore, their “predictions” have little implications for preventing the defects. This paper proposes an approach, “Human-Error-based Defect Forecast” (HEDF), to forecasting the exact defects at early stages of software development, before the code is produced, through knowledge about the cognitive mechanisms that cause developers’ errors. This approach is based on a model of human error mechanisms underlying software defects: a defect is caused by an error-prone scenario triggering human error modes , which psychologists have observed to recur across diverse activities. Software defects can then be forecast by identifying such error-prone scenarios the in requirements and/or design documents. We assessed this approach empirically, with 55 programmers in a programming competition and four representative analysts serving as the users of the approach. Impressively, the approach was able to forecast, at the requirement phase, 75.7% of the defects later committed by all of the programmers. When considering just the defect forms , which may manifest as distinct defects even in the same program, the proposed method predicted 31.8% of them. This approach substantially improved the defect forecasting performances for analysts of various expertise, with a minimum of 100% improvement, compared to forecasts without the approach. If the forecast had been used to prevent the defects, it could have saved an estimated 46.2% of the debugging effort and increased the fraction of programmers delivering an acceptable program by 32.6%. The observed excellent performance of HEDF in forecasting (early at requirement stage) the exact forms and locations of defects that may be later introduced by developers into code makes it a promising candidate for preventing the defects, worthy of further study.Keywords
All Related Versions
Funding Information
- Foundation for Science and Technology (FCT), I.P./MCTES through National Funds (PIDDAC), within the Scope of CISUC Research and Development Unit (UIDB/00326/2020, UIDP/00326/2020)
This publication has 64 references indexed in Scilit:
- Time variance and defect prediction in software projectsEmpirical Software Engineering, 2011
- What’s up with software metrics? – A preliminary mapping studyJournal of Systems and Software, 2010
- Do too many cooks spoil the broth? Using the number of developers to enhance defect prediction modelsEmpirical Software Engineering, 2008
- Requirements for tools for ambiguity identification and measurement in natural language requirements specificationsRequirements Engineering, 2008
- Predicting defect-prone software modules using support vector machinesJournal of Systems and Software, 2008
- A Reconsideration of Cognitive Load TheoryEducational Psychology Review, 2007
- View merging in the presence of incompleteness and inconsistencyRequirements Engineering, 2006
- The prediction of faulty classes using object-oriented design metricsJournal of Systems and Software, 2001
- A Working Memory Model of a Common Procedural ErrorCognitive Science, 1997
- Quantitative and qualitative differences between experts and novices in chunking computer software knowledgeInternational Journal of Human–Computer Interaction, 1994