Juliana Alves Pereira
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Juliana Alves Pereira has a total of 85 co-authors. Here are the top co-authors by number of shared publications:
- Alessandro Garcia
Pontifical Catholic University of Rio de Janeiro, Rio de Janeiro, Brazil
12 shared publications - Anderson Uchôa
Federal University of Ceará, Itapajé, Brazil
7 shared publications - Wesley Klewerton Guez Assuncao
Department of Computer Science, North Carolina State University, Raleigh, North Carolina, USA
5 shared publications - Daniel Coutinho
Pontifical Catholic University of Rio de Janeiro, Rio de Janeiro, Brazil
4 shared publications - Jessica Ribas
PUC-Rio
4 shared publications - Mathieu Acher
Univ Rennes, Inria, CNRS, IRISA
4 shared publications
Latest Publications
- Open Access
- 3 November 2026
- Published by Springer Nature in Empirical Software Engineering
- Vol. 32 (2) , 68
Women’s experiences and confidence in ICT education: an empirical study with 402 undergraduate students in Brazil
Context: Gender inequality remains a persistent issue in Information and Communication Technology (ICT) education. Although women represent less than 18% of ICT undergraduates in Brazil, most prior studies have focused on industry settings or early outreach, leaving a limited understanding of how undergraduate women perceive and experience inequality during their academic journey. Goal: This study investigates the perceptions of women undergraduate students in ICT programs in Brazil, aiming to identify the factors that influence their confidence, participation, and persistence, as well as the strategies that can support their permanence in the field. Method: We conducted a large-scale survey with 402 women from 19 Brazilian states and Federal District enrolled in diverse ICT courses. Given the voluntary recruitment strategy through academic and online networks, the sample primarily reflects the experiences of women who remained engaged in ICT programs and were reachable through these dissemination channels. Survey data were analyzed using complementary quantitative and qualitative techniques. Closed-ended questions were examined through statistical analyses, while open-ended responses were analyzed using grounded theory procedures. Notice that our findings may underestimate challenges faced by women who silently disengaged, transferred, or dropped out before being reached by the survey. Results: The findings indicate that participants frequently reported discriminatory practices from both peers and professors, including interruptions in class, sexist jokes, and underestimation of their technical abilities. More than 90% of participants perceived that gender stereotypes and lack of representation negatively affected their confidence and self-esteem, often leading to disengagement or course dropout. Greater exposure to women professors showed a statistically significant ordered association with participants’ comfort expressing opinions in their presence. Additionally, students highlighted strategies to boost confidence, such as mentorship programs, support groups, and institutional policies against harassment. Conclusions: Women’s underrepresentation in ICT programs is reinforced by systemic barriers that manifest in academic environments, generating isolation, alienation, and mistrust. These challenges are not isolated incidents but reflect deep-rooted sociocultural biases. Addressing them requires concrete institutional actions that promote safe, inclusive, and supportive learning environments, thereby strengthening women’s confidence and persistence in ICT education.
- 3 November 2026
- Published by Springer Nature in Empirical Software Engineering
- Vol. 32 (2) , 68
Open AccessWomen’s experiences and confidence in ICT education: an empirical study with 402 undergraduate students in Brazil
Context: Gender inequality remains a persistent issue in Information and Communication Technology (ICT) education. Although women represent less than 18% of ICT undergraduates in Brazil, most prior studies have focused on industry settings or early outreach, leaving a limited understanding of how undergraduate women perceive and experience inequality during their academic journey. Goal: This study investigates the perceptions of women undergraduate students in ICT programs in Brazil, aiming to identify the factors that influence their confidence, participation, and persistence, as well as the strategies that can support their permanence in the field. Method: We conducted a large-scale survey with 402 women from 19 Brazilian states and Federal District enrolled in diverse ICT courses. Given the voluntary recruitment strategy through academic and online networks, the sample primarily reflects the experiences of women who remained engaged in ICT programs and were reachable through these dissemination channels. Survey data were analyzed using complementary quantitative and qualitative techniques. Closed-ended questions were examined through statistical analyses, while open-ended responses were analyzed using grounded theory procedures. Notice that our findings may underestimate challenges faced by women who silently disengaged, transferred, or dropped out before being reached by the survey. Results: The findings indicate that participants frequently reported discriminatory practices from both peers and professors, including interruptions in class, sexist jokes, and underestimation of their technical abilities. More than 90% of participants perceived that gender stereotypes and lack of representation negatively affected their confidence and self-esteem, often leading to disengagement or course dropout. Greater exposure to women professors showed a statistically significant ordered association with participants’ comfort expressing opinions in their presence. Additionally, students highlighted strategies to boost confidence, such as mentorship programs, support groups, and institutional policies against harassment. Conclusions: Women’s underrepresentation in ICT programs is reinforced by systemic barriers that manifest in academic environments, generating isolation, alienation, and mistrust. These challenges are not isolated incidents but reflect deep-rooted sociocultural biases. Addressing them requires concrete institutional actions that promote safe, inclusive, and supportive learning environments, thereby strengthening women’s confidence and persistence in ICT education.
- Open Access
- 25 May 2026
- Published by Springer Nature in Empirical Software Engineering
- Vol. 31 (6)
Understanding developer well-being: measuring mental health and productivity in software teams
The productivity of software developers is influenced by various factors encompassing technical, organizational, and individual aspects. Among these factors, mental health has emerged as a critical element for sustaining performance and well-being in this context. This work primary focus lies in understanding developers’ perceptions and acceptance of metrics related to mental health and productivity. Firstly, a mapping study was conducted to map the existing knowledge in this area. A total of 178 papers were retrieved using a predefined search string. After applying strict inclusion and exclusion criteria, five secondary studies were selected and 99 factors influencing productivity and/or mental health in the workplace were identified. Secondly, a metrics catalog was developed based on these influencing factors, combining established indicators from literature with new metrics designed to monitor developers’ mental health and productivity. The catalog includes 12 metrics along with their respective measurement formulas. Thirdly, an assessment of this catalog was performed through a survey and structured interviews with industry professionals, gathering insights on the applicability and relevance of the proposed metrics. The survey was completed by 47 software developers, 22 of whom also participated in the interviews. Our results reveal that while developers largely recognized the value of the catalog, resistance emerged when these metrics were positioned as evaluative mechanisms in the workplace.
- 25 May 2026
- Published by Springer Nature in Empirical Software Engineering
- Vol. 31 (6)
Open AccessUnderstanding developer well-being: measuring mental health and productivity in software teams
The productivity of software developers is influenced by various factors encompassing technical, organizational, and individual aspects. Among these factors, mental health has emerged as a critical element for sustaining performance and well-being in this context. This work primary focus lies in understanding developers’ perceptions and acceptance of metrics related to mental health and productivity. Firstly, a mapping study was conducted to map the existing knowledge in this area. A total of 178 papers were retrieved using a predefined search string. After applying strict inclusion and exclusion criteria, five secondary studies were selected and 99 factors influencing productivity and/or mental health in the workplace were identified. Secondly, a metrics catalog was developed based on these influencing factors, combining established indicators from literature with new metrics designed to monitor developers’ mental health and productivity. The catalog includes 12 metrics along with their respective measurement formulas. Thirdly, an assessment of this catalog was performed through a survey and structured interviews with industry professionals, gathering insights on the applicability and relevance of the proposed metrics. The survey was completed by 47 software developers, 22 of whom also participated in the interviews. Our results reveal that while developers largely recognized the value of the catalog, resistance emerged when these metrics were positioned as evaluative mechanisms in the workplace.
- Open Access
- 9 May 2026
- Published by Springer Nature in Empirical Software Engineering
- Vol. 31 (5)
Leveraging large language models for sentiment analysis in GitHub pull request discussions
Social coding platforms like GitHub facilitate collaborative software development through pull requests (PRs), which generate discussions that significantly impact code quality, requirements, and design. Such conversations become a rich source of insights for improving development practices and predicting project outcomes and are subject to several human aspects that have been linked to code quality and PR acceptance. Sentiment analysis is one of the many ways to try to understand these human aspects. However, PR discussions are multifaceted, often involving technical jargon and aspects which limits the utility of general-purpose sentiment analysis tools. This has led to the creation of SE-specific tools, but recent studies have also observed that they demonstrate limited effectiveness. Thus, this study explores the potential of using large language models (LLMs) for this purpose, given their enhanced contextual understanding and ability to process technical language. We evaluated ten LLMs across proprietary and open-source categories, using two complementary datasets: a curated Gold dataset and the PRemo dataset, which captures real-world PR discussions. The models were assessed under zero-shot, few-shot and chain-of-thought prompting techniques on 8,913 messages. In addition, we establish baselines by evaluating fine-tuned transformer-based models. Results show that GPT-4o achieved the highest overall performance across the LLMs, though smaller models, such as Mistral Small and Deepseek-R1 32B delivered competitive results. Transformer-based models achieved excellent performance on the Gold dataset but exhibited degradation on the PRemo dataset. Finally, we conducted a qualitative analysis of misclassified instances, revealing recurring challenges related to technical terminology, sentiment-charged keywords, message length, and contextual ambiguity. These findings suggest that model selection should balance performance requirements against practical constraints, rather than defaulting to the largest available models.
- 9 May 2026
- Published by Springer Nature in Empirical Software Engineering
- Vol. 31 (5)
Open AccessLeveraging large language models for sentiment analysis in GitHub pull request discussions
Social coding platforms like GitHub facilitate collaborative software development through pull requests (PRs), which generate discussions that significantly impact code quality, requirements, and design. Such conversations become a rich source of insights for improving development practices and predicting project outcomes and are subject to several human aspects that have been linked to code quality and PR acceptance. Sentiment analysis is one of the many ways to try to understand these human aspects. However, PR discussions are multifaceted, often involving technical jargon and aspects which limits the utility of general-purpose sentiment analysis tools. This has led to the creation of SE-specific tools, but recent studies have also observed that they demonstrate limited effectiveness. Thus, this study explores the potential of using large language models (LLMs) for this purpose, given their enhanced contextual understanding and ability to process technical language. We evaluated ten LLMs across proprietary and open-source categories, using two complementary datasets: a curated Gold dataset and the PRemo dataset, which captures real-world PR discussions. The models were assessed under zero-shot, few-shot and chain-of-thought prompting techniques on 8,913 messages. In addition, we establish baselines by evaluating fine-tuned transformer-based models. Results show that GPT-4o achieved the highest overall performance across the LLMs, though smaller models, such as Mistral Small and Deepseek-R1 32B delivered competitive results. Transformer-based models achieved excellent performance on the Gold dataset but exhibited degradation on the PRemo dataset. Finally, we conducted a qualitative analysis of misclassified instances, revealing recurring challenges related to technical terminology, sentiment-charged keywords, message length, and contextual ambiguity. These findings suggest that model selection should balance performance requirements against practical constraints, rather than defaulting to the largest available models.
- Open Access
- 31 March 2026
- Published by Sociedade Brasileira de Computação in Journal of Software Engineering Research and Development
- Vol. 14 (1) , 304-318
On the Effectiveness of Feedforward Neural Networks for Performance Prediction in Highly Configurable Systems: The Case of Linux Kernel Size
Highly configurable software systems, such as the Linux kernel, expose thousands of configuration options whose combinations can drastically influence non-functional properties. In embedded or resource-constrained environments, kernel binary size is particularly important because it directly affects memory footprint, boot time, storage usage, and overall system responsiveness. Accurately predicting binary size from configuration options enables developers to evaluate trade-offs early in the development cycle, reducing the need for costly trial-and-error compilation and measurement processes. This study investigates the use of Feedforward Neural Networks (FNNs) for predicting Linux kernel binary size and compares their performance against strong classical Machine Learning (ML) baselines, including Random Forest and Gradient Boosting Trees. In addition, we analyze the impact of feature selection (FS) strategies on predictive accuracy and computational cost. We evaluate five FS approaches, including supervised tree-based importance rankings and a semantic, label-free method based on Word2Vec embeddings extracted from Linux kernel documentation. Our experiments were conducted on a dataset containing 9,670 Linux kernel configuration options. The results show that classical tree-based ensemble methods outperformed FNNs in predictive accuracy, with Gradient Boosting Trees achieving the best overall results (MAPE of 5.21%). Although FNNs combined with feature selection achieved reasonable accuracy (best MAPE of 8.26%) and benefited from reduced training times, they did not surpass the traditional ML baselines. These findings provide important empirical evidence that more complex neural architectures do not necessarily yield superior performance for SPL prediction tasks, even in high-dimensional configuration spaces. We also show that the Word2Vec-based semantic FS method offers a practical label-free alternative for early-stage scenarios where measured NFP data are unavailable or expensive to obtain, although it generally underperforms supervised feature selection strategies in both accuracy and efficiency. Overall, our findings provide practical guidance for SPL practitioners and researchers in selecting prediction models and preprocessing strategies that balance accuracy, interpretability, and computational cost in highly configurable systems.
- 31 March 2026
- Published by Sociedade Brasileira de Computação in Journal of Software Engineering Research and Development
- Vol. 14 (1) , 304-318
Open AccessOn the Effectiveness of Feedforward Neural Networks for Performance Prediction in Highly Configurable Systems: The Case of Linux Kernel Size
Highly configurable software systems, such as the Linux kernel, expose thousands of configuration options whose combinations can drastically influence non-functional properties. In embedded or resource-constrained environments, kernel binary size is particularly important because it directly affects memory footprint, boot time, storage usage, and overall system responsiveness. Accurately predicting binary size from configuration options enables developers to evaluate trade-offs early in the development cycle, reducing the need for costly trial-and-error compilation and measurement processes. This study investigates the use of Feedforward Neural Networks (FNNs) for predicting Linux kernel binary size and compares their performance against strong classical Machine Learning (ML) baselines, including Random Forest and Gradient Boosting Trees. In addition, we analyze the impact of feature selection (FS) strategies on predictive accuracy and computational cost. We evaluate five FS approaches, including supervised tree-based importance rankings and a semantic, label-free method based on Word2Vec embeddings extracted from Linux kernel documentation. Our experiments were conducted on a dataset containing 9,670 Linux kernel configuration options. The results show that classical tree-based ensemble methods outperformed FNNs in predictive accuracy, with Gradient Boosting Trees achieving the best overall results (MAPE of 5.21%). Although FNNs combined with feature selection achieved reasonable accuracy (best MAPE of 8.26%) and benefited from reduced training times, they did not surpass the traditional ML baselines. These findings provide important empirical evidence that more complex neural architectures do not necessarily yield superior performance for SPL prediction tasks, even in high-dimensional configuration spaces. We also show that the Word2Vec-based semantic FS method offers a practical label-free alternative for early-stage scenarios where measured NFP data are unavailable or expensive to obtain, although it generally underperforms supervised feature selection strategies in both accuracy and efficiency. Overall, our findings provide practical guidance for SPL practitioners and researchers in selecting prediction models and preprocessing strategies that balance accuracy, interpretability, and computational cost in highly configurable systems.
- Open Access
- 22 September 2025
- Published by Sociedade Brasileira de Computação
- p. 326-336
PRemo: A Dataset of Emotions Found on Pull Request Discussions
In software engineering, effective communication is key to ensuring software quality. On GitHub projects, one of the primary channels for such communication is pull request discussions. These discussions often contain high-level design decisions. Understanding communication dynamics within development teams is crucial in software engineering, and sentiments and emotions play a significant role in this context. While there are datasets available for sentiment analysis in this field, those focusing on specific emotions are rare, and some contexts, such as pull request discussions, remain underrepresented. To address these gaps, we propose a novel methodology for capturing sentiments and emotions in contextspecific data, resulting in the creation of PRemo. PRemo includes ≈1.8K manually labeled pull-request messages from 36 active opensource industry-relevant projects. It provides data on individual emotions (and their intensity), the surrounding context, and evaluator confidence. Built using a robust triple validation and two-pass labeling process, the dataset leverages an established psychological emotion model. Already applied in prior research, PRemo is a valuable resource for advancing emotion analysis in software engineering.
- 22 September 2025
- Published by Sociedade Brasileira de Computação
- p. 326-336
Open AccessPRemo: A Dataset of Emotions Found on Pull Request Discussions
In software engineering, effective communication is key to ensuring software quality. On GitHub projects, one of the primary channels for such communication is pull request discussions. These discussions often contain high-level design decisions. Understanding communication dynamics within development teams is crucial in software engineering, and sentiments and emotions play a significant role in this context. While there are datasets available for sentiment analysis in this field, those focusing on specific emotions are rare, and some contexts, such as pull request discussions, remain underrepresented. To address these gaps, we propose a novel methodology for capturing sentiments and emotions in contextspecific data, resulting in the creation of PRemo. PRemo includes ≈1.8K manually labeled pull-request messages from 36 active opensource industry-relevant projects. It provides data on individual emotions (and their intensity), the surrounding context, and evaluator confidence. Built using a robust triple validation and two-pass labeling process, the dataset leverages an established psychological emotion model. Already applied in prior research, PRemo is a valuable resource for advancing emotion analysis in software engineering.
- Open Access
- 22 September 2025
- Published by Sociedade Brasileira de Computação
- p. 315-325
Evaluating the Potential of Large Language Models in Security-Related Software Requirements Classification
An effective classification of security-related software requirements is crucial to mitigate potential threats and ensure robust system design. This study investigates the performance of Large Language Models (LLMs) in classifying security-related requirements compared to traditional Machine Learning (ML) methods. Using the SecReq, DOSSPRE and PROMISE+ datasets, we evaluated ten LLMs across various prompt engineering strategies. The results demonstrate that LLMs achieve high accuracy and outperform traditional ML-based models in several evaluation scenarios and that prompt engineering can significantly enhance the model’s ability to identify security-related requirements. This work underscores the domaingeneralization capabilities of LLMs and their potential to streamline requirements classification without the complexity of feature engineering or dataset-specific fine-tuning often required by ML-based approaches. Researchers, practitioners, and tool developers can leverage these findings to advance automated approaches in security requirements engineering.
- 22 September 2025
- Published by Sociedade Brasileira de Computação
- p. 315-325
Open AccessEvaluating the Potential of Large Language Models in Security-Related Software Requirements Classification
An effective classification of security-related software requirements is crucial to mitigate potential threats and ensure robust system design. This study investigates the performance of Large Language Models (LLMs) in classifying security-related requirements compared to traditional Machine Learning (ML) methods. Using the SecReq, DOSSPRE and PROMISE+ datasets, we evaluated ten LLMs across various prompt engineering strategies. The results demonstrate that LLMs achieve high accuracy and outperform traditional ML-based models in several evaluation scenarios and that prompt engineering can significantly enhance the model’s ability to identify security-related requirements. This work underscores the domaingeneralization capabilities of LLMs and their potential to streamline requirements classification without the complexity of feature engineering or dataset-specific fine-tuning often required by ML-based approaches. Researchers, practitioners, and tool developers can leverage these findings to advance automated approaches in security requirements engineering.
- Open Access
- 22 September 2025
- Published by Sociedade Brasileira de Computação
- p. 371-381
Diversity Matters: Perceived Inclusion and Discrimination by Brazilian Tech Professionals
Diversity and inclusion are widely recognized as essential drivers of innovation in software engineering. However, systemic inequalities and implicit biases remain persistent challenges. This study aims to explore how tech professionals in Brazil perceive inclusion, discrimination, and unconscious bias in the workplace. We surveyed 220 tech professionals, focusing on experiences of bias, perceptions of fairness, and the influence of organizational setting. The findings reveal that 95.2% of women experienced sexism, compared to 70.3% of men; 77.1% of non-white professionals identify racism as a key issue, while 61.5% of disabled professionals report experiencing ableism. Hybrid work models were associated with the highest reports of sexism (80%), while employees at smaller companies reported more frequent instances of ableism than those at larger ones. Despite widespread acknowledgment of discrimination, only 23% of respondents explicitly recognized the presence of unconscious bias in themselves. While most see their teams as diverse, leadership inclusion lags, especially for disabled (25.9%) and neurodivergent (36.4%) professionals. These results reveal persistent equity gaps and call for DEI (Diversity, Equity, and Inclusion) strategies in Brazil’s tech sector.
- 22 September 2025
- Published by Sociedade Brasileira de Computação
- p. 371-381
Open AccessDiversity Matters: Perceived Inclusion and Discrimination by Brazilian Tech Professionals
Diversity and inclusion are widely recognized as essential drivers of innovation in software engineering. However, systemic inequalities and implicit biases remain persistent challenges. This study aims to explore how tech professionals in Brazil perceive inclusion, discrimination, and unconscious bias in the workplace. We surveyed 220 tech professionals, focusing on experiences of bias, perceptions of fairness, and the influence of organizational setting. The findings reveal that 95.2% of women experienced sexism, compared to 70.3% of men; 77.1% of non-white professionals identify racism as a key issue, while 61.5% of disabled professionals report experiencing ableism. Hybrid work models were associated with the highest reports of sexism (80%), while employees at smaller companies reported more frequent instances of ableism than those at larger ones. Despite widespread acknowledgment of discrimination, only 23% of respondents explicitly recognized the presence of unconscious bias in themselves. While most see their teams as diverse, leadership inclusion lags, especially for disabled (25.9%) and neurodivergent (36.4%) professionals. These results reveal persistent equity gaps and call for DEI (Diversity, Equity, and Inclusion) strategies in Brazil’s tech sector.
- Open Access
- 22 September 2025
- Published by Sociedade Brasileira de Computação
- p. 304-314
404: Civility Not Found? Evaluating the Effectiveness of Small Language Models in Detecting Incivility in GitHub Conversations
Context: Incivility in open-source software (OSS) platforms like GitHub can harm collaboration, discourage contributor participation, and impact code quality. Although current moderation tools based on Machine Learning (ML) and Natural Language Processing (NLP) offer some support, they often struggle to detect nuanced or implicit types of incivility. Goal: This study aims to assess the effectiveness of Small Language Models (SLMs) in detecting both coarse-grained (civil vs. uncivil) and fine-grained (specific types) incivility in GitHub conversations (issues and pull requests), and to understand how different prompting strategies influence detection performance. Method: We evaluate ten SLMs (3B-14B parameters) across five prompt strategies, on a labeled dataset with more than 6k GitHub conversations. We also compare the best-performing SLMs with five traditional ML models using two text-encoding techniques. Results: Our results reveal that SLMs perform well in detecting civil comments, but their effectiveness in detecting uncivil comments depends on model size. Models with 9B+ parameters (e.g., deepseek-r1, gpt-4o-mini) show improved performance on uncivil comments. For the fine-grained granularity, prompting strategy plays a critical role, with role-based prompting achieving the best results, particularly for implicit incivility types (e.g., Irony and Mocking), even when SLMs struggle with these types of incivility. Traditional ML models still perform well in explicit cases like Threat and Insulting. Conclusion: Our findings highlight the effectiveness of SLMs and prompt strategies in enhancing the detection of incivility within collaborative software development settings.
- 22 September 2025
- Published by Sociedade Brasileira de Computação
- p. 304-314
Open Access404: Civility Not Found? Evaluating the Effectiveness of Small Language Models in Detecting Incivility in GitHub Conversations
Context: Incivility in open-source software (OSS) platforms like GitHub can harm collaboration, discourage contributor participation, and impact code quality. Although current moderation tools based on Machine Learning (ML) and Natural Language Processing (NLP) offer some support, they often struggle to detect nuanced or implicit types of incivility. Goal: This study aims to assess the effectiveness of Small Language Models (SLMs) in detecting both coarse-grained (civil vs. uncivil) and fine-grained (specific types) incivility in GitHub conversations (issues and pull requests), and to understand how different prompting strategies influence detection performance. Method: We evaluate ten SLMs (3B-14B parameters) across five prompt strategies, on a labeled dataset with more than 6k GitHub conversations. We also compare the best-performing SLMs with five traditional ML models using two text-encoding techniques. Results: Our results reveal that SLMs perform well in detecting civil comments, but their effectiveness in detecting uncivil comments depends on model size. Models with 9B+ parameters (e.g., deepseek-r1, gpt-4o-mini) show improved performance on uncivil comments. For the fine-grained granularity, prompting strategy plays a critical role, with role-based prompting achieving the best results, particularly for implicit incivility types (e.g., Irony and Mocking), even when SLMs struggle with these types of incivility. Traditional ML models still perform well in explicit cases like Threat and Insulting. Conclusion: Our findings highlight the effectiveness of SLMs and prompt strategies in enhancing the detection of incivility within collaborative software development settings.
- Open Access
- 22 September 2025
- Published by Sociedade Brasileira de Computação
- p. 852-858
Comparing LLMs and Proposing an ML-Based Approach for Search String Generation in Systematic Literature Reviews
The formulation of an effective search string is a critical process in systematic literature reviews (SLRs), as it directly influences both the coverage and precision of the retrieved studies. Traditionally, this process relies on manual keyword selection and expert-driven refinements, making it laborious, susceptible to human bias, and often inaccessible to non-specialists. To address these limitations, this study explores the application of artificial intelligence (AI) to support the generation of search strings.We organized our ongoing investigation into two main phases. In the first phase, we evaluated the performance of search strings generated by different large language models (LLMs), specifically Llama-8B, Gemma-12B, and Mistral-Nemo-12B, using a previously published SLR as a benchmark. Our results suggest that, while LLMs can assist in search string formulation, their effectiveness is inconsistent and sensitive to input conditions. Motivated by these limitations, we propose a semi-automated pipeline based on Machine Learning (ML). Through our preliminary analysis, we proposed a standardized reproducible evaluation framework to assess and compare AI-based search string generation strategies, including our proposed ML-based approach.
- 22 September 2025
- Published by Sociedade Brasileira de Computação
- p. 852-858
Open AccessComparing LLMs and Proposing an ML-Based Approach for Search String Generation in Systematic Literature Reviews
The formulation of an effective search string is a critical process in systematic literature reviews (SLRs), as it directly influences both the coverage and precision of the retrieved studies. Traditionally, this process relies on manual keyword selection and expert-driven refinements, making it laborious, susceptible to human bias, and often inaccessible to non-specialists. To address these limitations, this study explores the application of artificial intelligence (AI) to support the generation of search strings.We organized our ongoing investigation into two main phases. In the first phase, we evaluated the performance of search strings generated by different large language models (LLMs), specifically Llama-8B, Gemma-12B, and Mistral-Nemo-12B, using a previously published SLR as a benchmark. Our results suggest that, while LLMs can assist in search string formulation, their effectiveness is inconsistent and sensitive to input conditions. Motivated by these limitations, we propose a semi-automated pipeline based on Machine Learning (ML). Through our preliminary analysis, we proposed a standardized reproducible evaluation framework to assess and compare AI-based search string generation strategies, including our proposed ML-based approach.
- Open Access
- 22 September 2025
- Published by Sociedade Brasileira de Computação
- p. 762-768
Toward Reliable Forward Snowballing in Systematic Literature Reviews: A Comparative Study and Framework Proposal
Systematic Literature Reviews (SLRs) play a vital role in the software engineering field by synthesizing existing knowledge, identifying research gaps, and guiding future investigations with methodological rigor. Given the rapid growth of published research, techniques such as snowballing are essential. Forward snowballing, in particular, helps to discover newer studies that cite key seed papers, enhancing the completeness of SLRs. Despite its value, when performed manually, it remains a labor-intensive and error-prone task. To address this challenge, we mapped existing tools that support forward snowballing. Our analysis focuses on the capabilities of these tools to automate core tasks, including the identification of relevant articles and the extraction of bibliographic metadata. We critically examine their reliability by comparing their outputs against a manually curated forward snowballing process. Key evaluation criteria include the completeness and relevance of the metadata retrieved, the quality of the retrieved scientific databases, and the ease of integration into SLR workflows. As future research, based on our findings, we will propose a framework that takes advantage of existing tools to automate forward snowballing.
- 22 September 2025
- Published by Sociedade Brasileira de Computação
- p. 762-768
Open AccessToward Reliable Forward Snowballing in Systematic Literature Reviews: A Comparative Study and Framework Proposal
Systematic Literature Reviews (SLRs) play a vital role in the software engineering field by synthesizing existing knowledge, identifying research gaps, and guiding future investigations with methodological rigor. Given the rapid growth of published research, techniques such as snowballing are essential. Forward snowballing, in particular, helps to discover newer studies that cite key seed papers, enhancing the completeness of SLRs. Despite its value, when performed manually, it remains a labor-intensive and error-prone task. To address this challenge, we mapped existing tools that support forward snowballing. Our analysis focuses on the capabilities of these tools to automate core tasks, including the identification of relevant articles and the extraction of bibliographic metadata. We critically examine their reliability by comparing their outputs against a manually curated forward snowballing process. Key evaluation criteria include the completeness and relevance of the metadata retrieved, the quality of the retrieved scientific databases, and the ease of integration into SLR workflows. As future research, based on our findings, we will propose a framework that takes advantage of existing tools to automate forward snowballing.