TANDEM: A Taxonomy and a Dataset of Real-World Performance Bugs
Open Access
- 8 June 2020
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 8, 107214-107228
Abstract
The detection of performance bugs, like those causing an unexpected execution time, has gained much attention in the last years due to their potential impact in safety-critical and resource-constrained applications. Much effort has been put on trying to understand the nature of performance bugs in different domains as a starting point for the development of effective testing techniques. However, the lack of a widely accepted classification scheme of performance faults and, more importantly, the lack of well-documented and understandable datasets makes it difficult to draw rigorous and verifiable conclusions widely accepted by the community. In this paper, we present TANDEM, a dual contribution related to real-world performance bugs. Firstly, we propose a taxonomy of performance bugs based on a thorough systematic review of the related literature, divided into three main categories: effects, causes and contexts of bugs. Secondly, we provide a complete collection of fully documented real-world performance bugs. Together, these contributions pave the way for the development of stronger and reproducible research results on performance testing.Keywords
Funding Information
- European Commission
- Spanish Ministry of Science, Innovation and Universities (RTI2018-101204-B-C21, RTI2018-093608-B-C33)
- Andalusian Research, Development and Innovation Program (US-1264651)
This publication has 76 references indexed in Scilit:
- MemInsight: platform-independent memory debugging for JavaScript2015
- What change history tells us about thread synchronization2015
- Static detection of asymptotic performance bugs in collection traversals2015
- CARAMEL: Detecting and Fixing Performance Problems That Have Non-Intrusive Fixes2015
- Poster: Automatically Fixing Real-World JavaScript Performance Bugs2015
- Diagnosing Energy Efficiency and Performance for Mobile Internetware ApplicationsIEEE Software, 2015
- GreenDroid: Automated Diagnosis of Energy Inefficiency for Smartphone ApplicationsIEEE Transactions on Software Engineering, 2014
- Model checking driven static analysis for the real world: designing and tuning large scale bug detectionInnovations in Systems and Software Engineering, 2012
- LeakChaserACM SIGPLAN Notices, 2011
- Software errors and complexity: an empirical investigation0Communications of the ACM, 1984