Improving the Multi-Class Classification of Non-Functional Requirements in Spanish: A Study of Dataset Balancing and Performance
Open Access
- 2 November 2025
- journal article
- research article
- Published by Springer Nature in Empirical Software Engineering
- Vol. 31 (1) , 1-55
Abstract
Context: In recent years, the multi-class classification of non-functional requirements has seen improvements through the use of Machine Learning algorithms. However, challenges such as data scarcity and class imbalance persist, particularly for languages other than English, such as Spanish. Objective: This study aims to analyze the performance metrics of Machine Learning algorithms for classifying non-functional requirements translated into and originally written in Spanish. It evaluates the effectiveness of dataset balancing techniques and conducts cross-dataset validation to assess the generalizability of the models. Method: A dataset balancing process was conducted using a combination of oversampling and undersampling techniques. Six algorithms were trained in two experiments using a hyperparameter tuning process, employing two different datasets: PROMISE_exp_translated and the newly PROMISE_exp_balanced . The best-performing models were further tested on unseen data to evaluate their generalizability. Results: Logistic Regression and Naive Bayes demonstrated superior performance on the translated dataset, achieving f1-scores of 82% and 81%, respectively. Although overall performance decreased on the balanced dataset, specific underrepresented classes such as Portability and Fault Tolerance benefited from the balancing process. Conclusion: Shallow Machine Learning algorithms are effective for classifying Spanish non-functional requirements, particularly when addressing data imbalance. The study highlights the importance of dataset balancing in improving classification performance for specific classes and provides insights into the challenges of generalizing models across datasets.Keywords
Funding Information
- Universidade da Coruña
This publication has 88 references indexed in Scilit:
- ChatGPT Prompt Patterns for Improving Code Quality, Refactoring, Requirements Elicitation, and Software Design2024
- The application of AI techniques in requirements classification: a systematic mappingArtificial Intelligence Review, 2024
- Requirements Classification Using FastText and BETO in Spanish Documents2023
- Improving Multi-class Text Classification Using Balancing Techniques2023
- Software Functional Requirements Classification Using Ensemble Learning2022
- Empirical evaluation of tools for hairy requirements engineering tasksEmpirical Software Engineering, 2021
- Pre-trained Data Augmentation for Text Classification2020
- Survey on supervised machine learning techniques for automatic text classificationArtificial Intelligence Review, 2019
- Study of Various Classifiers for Identification and Classification of Non-functional Requirements2018
- Linear Classification and Regression for Text2018