A General Testability Theory: Classes, Properties, Complexity, and Testing Reductions
- 18 June 2014
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Software Engineering
- Vol. 40 (9) , 862-894
Abstract
In this paper we develop a general framework to reason about testing. The difficulty of testing is assessed in terms of the amount of tests that must be applied to determine whether the system is correct or not. Based on this criterion, five testability classes are presented and related. We also explore conditions that enable and disable finite testability, and their relation to testing hypotheses is studied. We measure how far incomplete test suites are from being complete, which allows us to compare and select better incomplete test suites. The complexity of finding that measure, as well as the complexity of finding minimum complete test suites, is identified. Furthermore, we address the reduction of testing problems to each other, that is, we study how the problem of finding test suites to test systems of some kind can be reduced to the problem of finding test suites for another kind of systems. This enables to export testing methods. In order to illustrate how general notions are applied to specific cases, many typical examples from the formal testing techniques domain are presented.Keywords
This publication has 26 references indexed in Scilit:
- FSM-based conformance testing methods: A survey annotated with experimental evaluationInformation and Software Technology, 2010
- Verdict functions in testing with a fault domain or test hypothesesACM Transactions on Software Engineering and Methodology, 2009
- Prioritizing JUnit Test Cases: An Empirical Assessment and Cost-Benefits AnalysisEmpirical Software Engineering, 2006
- Comparing test sets and criteria in the presence of test hypotheses and fault domainsACM Transactions on Software Engineering and Methodology, 2002
- A methodology of testing high-level Petri netsInformation and Software Technology, 2002
- Testing timed automataTheoretical Computer Science, 2001
- Conformance testing with labelled transition systems: Implementation relations and test generationComputer Networks and ISDN Systems, 1996
- Using Vapnik–Chervonenkis Dimension to Analyze the Testing Complexity of Program SegmentsInformation and Computation, 1996
- Metric Space-based Test-data Adequacy CriteriaThe Computer Journal, 1988
- Hints on Test Data Selection: Help for the Practicing ProgrammerComputer, 1978