Semi-supervised Crowdsourced Test Report Clustering via Screenshot-Text Binding Rules
Open Access
- 12 July 2024
- journal article
- research article
- Published by Association for Computing Machinery (ACM) in Proceedings of the ACM on Software Engineering
- Vol. 1 (FSE) , 1540-1563
Abstract
Due to the openness of the crowdsourced testing paradigm, crowdworkers submit massive spotty duplicate test reports, which hinders developers from effectively reviewing the reports and detecting bugs. Test report clustering is widely used to alleviate this problem and improve the effectiveness of crowdsourced testing. Existing clustering methods basically rely on the analysis of textual descriptions. A few methods are independently supplemented by analyzing screenshots in test reports as pixel sets, leaving out the semantics of app screenshots from the widget perspective. Further, ignoring the semantic relationships between screenshots and textual descriptions may lead to the imprecise analysis of test reports, which in turn negatively affects the clustering effectiveness. This paper proposes a semi-supervised crowdsourced test report clustering approach, namely SemCluster . SemCluster respectively extracts features from app screenshots and textual descriptions and forms the structure feature, the content feature, the bug feature, and reproduction steps. The clustering is principally conducted on the basis of the four features. Further, in order to avoid bias of specific individual features, SemCluster exploits the semantic relationships between app screenshots and textual descriptions to form the semantic binding rules as guidance for clustering crowdsourced test reports. Experiment results show that SemCluster outperforms state-of-the-art approaches on six widely used metrics by 10.49% – 200.67%, illustrating the excellent effectiveness.Keywords
Funding Information
- National Natural Science Foundation of China (61932012, 62141215, 62272220, 62372228)
- the Science, Technology, and Innovation Commission of Shenzhen Municipality (CJGJZD20200617103001003)
This publication has 58 references indexed in Scilit:
- Characterizing and Comparing External Measures for the Assessment of Cluster Analysis and Community DetectionIEEE Access, 2021
- A systemic framework for crowdsourced test report quality assessmentEmpirical Software Engineering, 2020
- Images don’t lie: Duplicate crowdtesting reports detection with screenshot informationInformation and Software Technology, 2019
- Successes, challenges, and rethinking – an industrial investigation on crowdsourced mobile application testingEmpirical Software Engineering, 2018
- A survey of the use of crowdsourcing in software engineeringJournal of Systems and Software, 2017
- Tree edit distance: Robust and memory-efficientInformation Systems, 2016
- A contextual approach towards more accurate duplicate bug report detection and rankingEmpirical Software Engineering, 2015
- Detecting Duplicate Bug Report Using Character N-Gram-Based Features2010
- Comparing partitionsJournal of Classification, 1985
- Who Belongs in the Family?Psychometrika, 1953