Profiling deployed software: assessing strategies and testing opportunities
Authors:
- 31 May 2005
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Software Engineering
- Vol. 31 (4) , 312-327
Abstract
An understanding of how software is employed in the field can yield many opportunities for quality improvements. Profiling released software can provide such an understanding. However, profiling released software is difficult due to the potentially large number of deployed sites that must be profiled, the transparency requirements at a user's site, and the remote data collection and deployment management process. Researchers have recently proposed various approaches to tap into the opportunities offered by profiling deployed systems and overcome those challenges. Initial studies have illustrated the application of these approaches and have shown their feasibility. Still, the proposed approaches, and the tradeoffs between overhead, accuracy, and potential benefits for the testing activity have been barely quantified. This paper aims to overcome those limitations. Our analysis of 1,200 user sessions on a 155 KLOC deployed system substantiates the ability of field data to support test suite improvements, assesses the efficiency of profiling techniques for released software, and the effectiveness of testing efforts that leverage profiled field data.Keywords
This publication has 28 references indexed in Scilit:
- Covering arrays for efficient fault characterization in complex configuration spacesIEEE Transactions on Software Engineering, 2006
- Skoll: distributed continuous quality assurance2004
- Active learning for automatic classification of software behavior2004
- Bug isolation via remote program sampling2003
- An approach to large-scale collection of application usage data over the internet2002
- Monitoring deployed software using software tomography2002
- Residual test coverage monitoring1999
- Software release managementACM SIGSOFT Software Engineering Notes, 1997
- Exploiting hardware performance counters with flow and context sensitive profilingACM SIGPLAN Notices, 1997
- GprofACM SIGPLAN Notices, 1982