Characterizing and Classifying Developer Forum Posts with their Intentions
Authors:
Preprint
Abstract
With the rapid growth of the developer community, the amount of posts on online technical forums has been growing rapidly, which poses difficulties for users to filter useful posts and find important information. Tags provide a concise feature dimension for users to locate their interested posts and for search engines to index the most relevant posts according to the queries. However, most tags are only focused on the technical perspective (e.g., program language, platform, tool). In most cases, forum posts in online developer communities reveal the author's intentions to solve a problem, ask for advice, share information, etc. The modeling of the intentions of posts can provide an extra dimension to the current tag taxonomy. By referencing previous studies and learning from industrial perspectives, we create a refined taxonomy for the intentions of technical forum posts. Through manual labeling and analysis on a sampled post dataset extracted from online forums, we understand the relevance between the constitution of posts (code, error messages) and their intentions. Furthermore, inspired by our manual study, we design a pre-trained transformer-based model to automatically predict post intentions. The best variant of our intention prediction framework, which achieves a Micro F1-score of 0.589, Top 1-3 accuracy of 62.6% to 87.8%, and an average AUC of 0.787, outperforms the state-of-the-art baseline approach. Our characterization and automated classification of forum posts regarding their intentions may help forum maintainers or third-party tool developers improve the organization and retrieval of posts on technical forums. We have released our annotated dataset and codes in our supplementary material package.Keywords
All Related Versions
This publication has 18 references indexed in Scilit:
- PTM4Tag+: Tag recommendation of stack overflow posts with pre-trained modelsEmpirical Software Engineering, 2024
- TagDC: A tag recommendation method for software information sites with a combination of deep learning and collaborative filteringJournal of Systems and Software, 2020
- Modeling stack overflow tags and topics as a hierarchy of conceptsJournal of Systems and Software, 2019
- What kind of questions do developers ask on Stack Overflow? A comparison of automated approaches to classify posts into question categoriesEmpirical Software Engineering, 2019
- FastTagRec: fast tag recommendation for software information sitesAutomated Software Engineering, 2018
- EnTagRec ++: An enhanced tag recommendation system for software information sitesEmpirical Software Engineering, 2017
- TagCombine: Recommending Tags to Contents in Software Information SitesJournal of Computer Science and Technology, 2015
- What are mobile developers asking about? A large scale study using stack overflowEmpirical Software Engineering, 2015
- What are developers talking about? An analysis of topics and trends in Stack OverflowEmpirical Software Engineering, 2012
- A Simple Generalisation of the Area Under the ROC Curve for Multiple Class Classification ProblemsMachine Learning, 2001