Studying the fault-detection effectiveness of GUI test cases for rapidly evolving software
Authors:
- 21 November 2005
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Software Engineering
- Vol. 31 (10) , 884-896
Abstract
Software is increasingly being developed/maintained by multiple, often geographically distributed developers working concurrently. Consequently, rapid-feedback-based quality assurance mechanisms such as daily builds and smoke regression tests, which help to detect and eliminate defects early during software development and maintenance, have become important. This paper addresses a major weakness of current smoke regression testing techniques, i.e., their inability to automatically (re)test graphical user interfaces (GUIs). Several contributions are made to the area of GUI smoke testing. First, the requirements for GUI smoke testing are identified and a GUI smoke test is formally defined as a specialized sequence of events. Second, a GUI smoke regression testing process called daily automated regression tester (DART) that automates GUI smoke testing is presented. Third, the interplay between several characteristics of GUI smoke test suites including their size, fault detection ability, and test oracles is empirically studied. The results show that: 1) the entire smoke testing process is feasible in terms of execution time, storage space, and manual effort, 2) smoke tests cannot cover certain parts of the application code, 3) having comprehensive test oracles may make up for not having long smoke test cases, and 4) using certain oracles can make up for not having large smoke test suites.Keywords
This publication has 14 references indexed in Scilit:
- User-based testing of GUI sequences and their interactions2005
- Automating regression testing for evolving GUI softwareJournal of Software Maintenance and Evolution: Research and Practice, 2005
- Empirical evaluation of the fault-detection effectiveness of smoke regression test cases for GUI-based software2004
- GUI ripping: reverse engineering of graphical user interfaces for testing2004
- What test oracle should I use for effective GUI testing?2004
- Daily build and feature development in large distributed projects2002
- Hierarchical GUI test case generation using automated planningIEEE Transactions on Software Engineering, 2001
- A Tool for Automatically Generating Transcripts of Human-Computer InteractionProceedings of the Human Factors and Ergonomics Society Annual Meeting, 1993
- An integrated general purpose automated test environment1993
- Applied Multivariate Data Analysis1991