ArchISMiner: A Framework for Automatic Mining of Architectural Issue-Solution Pairs from Online Developer Communities
Preprint
Abstract
Stack Overflow (SO), a leading online community forum, is a rich source of software development knowledge. However, locating architectural knowledge, such as architectural solutions remains challenging due to the overwhelming volume of unstructured content and fragmented discussions. Developers must manually sift through posts to find relevant architectural insights, which is time-consuming and error-prone. This study introduces ArchISMiner, a framework for mining architectural knowledge from SO. The framework comprises two complementary components: ArchPI and ArchISPE. ArchPI trains and evaluates multiple models, including conventional ML/DL models, Pre-trained Language Models (PLMs), and Large Language Models (LLMs), and selects the best-performing model to automatically identify Architecture-Related Posts (ARPs) among programming-related discussions. ArchISPE employs an indirect supervised approach that leverages diverse features, including BERT embeddings and local TextCNN features, to extract architectural issue-solution pairs. Our evaluation shows that the best model in ArchPI achieves an F1-score of 0.960 in ARP detection, and ArchISPE outperforms baselines in both SE and NLP fields, achieving F1-scores of 0.883 for architectural issues and 0.894 for solutions. A user study further validated the quality (e.g., relevance and usefulness) of the identified ARPs and the extracted issue-solution pairs. Moreover, we applied ArchISMiner to three additional forums, releasing a dataset of over 18K architectural issue-solution pairs. Overall, ArchISMiner can help architects and developers identify ARPs and extract succinct, relevant, and useful architectural knowledge from developer communities more accurately and efficiently. The replication package of this study has been provided at https://github.com/JeanMusenga/ArchISPEKeywords
All Related Versions
This publication has 34 references indexed in Scilit:
- How do users revise architectural related questions on stack overflow: an empirical studyEmpirical Software Engineering, 2025
- Mining architectural information: A systematic mapping studyEmpirical Software Engineering, 2024
- An empirical study of untangling patterns of two-class dependency cyclesEmpirical Software Engineering, 2024
- BTLink : automatic link recovery between issues and commits based on pre-trained BERT modelEmpirical Software Engineering, 2023
- Mining and relating design contexts and design patterns from Stack OverflowEmpirical Software Engineering, 2021
- Recurrent networks with attention and convolutional networks for sentence representation and classificationApplied Intelligence, 2018
- Recommending reference API documentationEmpirical Software Engineering, 2014
- What are developers talking about? An analysis of topics and trends in Stack OverflowEmpirical Software Engineering, 2012
- A Survey of Text Classification Algorithms2012
- Machine learning: a review of classification and combining techniquesArtificial Intelligence Review, 2006