The NLBSE'24 Tool Competition

Authors:
Open Access
Publisher website
Google Scholar
Abstract
We report on the organization and results of the tool competition of the third International Workshop on Natural Language-based Software Engineering (NLBSE'24). As in prior editions, we organized the competition on automated issue report classification, with focus on small repositories, and on automated code comment classification, with a larger dataset. In this tool competition edition, six teams submitted multiple classification models to automatically classify issue reports and code comments. The submitted models were fine-tuned and evaluated on a benchmark dataset of 3 thousand issue reports or 82 thousand code comments, respectively. This paper reports details of the competition, including the rules, the teams and contestant models, and the ranking of models based on their average classification performance across issue report and code comment types.
Funding Information
  • NSF (National Science Foundation) (CCF-1955853)
  • Horizon 2020 (957254-COSMOS)

This publication has 31 references indexed in Scilit: