Izabella Silva
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Izabella Silva has a total of 10 co-authors. Here are the top co-authors by number of shared publications:
- Danyllo Albuquerque
IFPB
3 shared publications - Kyller Gorgônio
Federal University of Campina Grande
3 shared publications - Angelo Perkusich
UFCG, Campina Grande, Brazil
2 shared publications - Mirko Perkusich
UFCG
2 shared publications Angelo Perkusich
UFCG
1 shared publication- Danilo F. S. Santos
UFCG
1 shared publication
Latest Publications
- Open Access
- 1 April 2026
- Published by Springer Nature
- p. 155-174
Evaluating the Quality of User Stories: An Extended Comparative Study of Multiple LLMs and Rule-Based Tools
Background: Ensuring the quality of user stories is vital to Agile Software Development. Rule-based tools like AQUSA, based on the Quality User Story (QUS) framework, offer reliable structural checks but struggle with context-sensitive or pragmatic issues. Large Language Models (LLMs) have emerged as potential alternatives, yet prior studies often rely on small datasets, older models, or lack direct comparison with rule-based baselines. Objective: This study aims to assess the effectiveness of modern LLMs relative to a rule-based tool (AQUSA) for detecting defects in user stories, considering both structural and contextual dimensions. Method: We conduct a large-scale comparative evaluation involving AQUSA and three GPT-family LLMs (GPT-5, GPT-5-mini, and GPT-4), using 182 user stories drawn from three industrial datasets. We apply both quantitative metrics (precision, recall, F1-score) and qualitative analysis of feedback clarity and defect relevance. Results: GPT-5-mini achieved the highest recall (0.81) and overall F1-score (0.62), while AQUSA attained the highest precision (0.61) with significantly fewer false positives. GPT-5 showed high hallucination rates and instability; GPT-4 was overly conservative, leading to under-detection of defects. Conclusion: Neither rule-based nor GPT-family LLM-based approaches suffice in isolation. Rule-based tools enforce structural rigor, while LLMs capture nuanced linguistic and pragmatic flaws. We advocate a hybrid “Dual-gate” strategy—using AQUSA for structural validation followed by lightweight LLMs for contextual refinement—to improve the reliability and scalability of user story quality assessment in agile environments.
- 1 April 2026
- Published by Springer Nature
- p. 155-174
Open AccessEvaluating the Quality of User Stories: An Extended Comparative Study of Multiple LLMs and Rule-Based Tools
Background: Ensuring the quality of user stories is vital to Agile Software Development. Rule-based tools like AQUSA, based on the Quality User Story (QUS) framework, offer reliable structural checks but struggle with context-sensitive or pragmatic issues. Large Language Models (LLMs) have emerged as potential alternatives, yet prior studies often rely on small datasets, older models, or lack direct comparison with rule-based baselines. Objective: This study aims to assess the effectiveness of modern LLMs relative to a rule-based tool (AQUSA) for detecting defects in user stories, considering both structural and contextual dimensions. Method: We conduct a large-scale comparative evaluation involving AQUSA and three GPT-family LLMs (GPT-5, GPT-5-mini, and GPT-4), using 182 user stories drawn from three industrial datasets. We apply both quantitative metrics (precision, recall, F1-score) and qualitative analysis of feedback clarity and defect relevance. Results: GPT-5-mini achieved the highest recall (0.81) and overall F1-score (0.62), while AQUSA attained the highest precision (0.61) with significantly fewer false positives. GPT-5 showed high hallucination rates and instability; GPT-4 was overly conservative, leading to under-detection of defects. Conclusion: Neither rule-based nor GPT-family LLM-based approaches suffice in isolation. Rule-based tools enforce structural rigor, while LLMs capture nuanced linguistic and pragmatic flaws. We advocate a hybrid “Dual-gate” strategy—using AQUSA for structural validation followed by lightweight LLMs for contextual refinement—to improve the reliability and scalability of user story quality assessment in agile environments.
- Open Access
- 23 September 2025
- Published by Sociedade Brasileira de Computação
- p. 25-30
How Software Testers Perceive Large Language Models: A Focus Group Study with Lexicometric Analysis
Context. Large Language Models (LLMs) are increasingly embedded in software-engineering toolchains, with potential to enhance quality-assurance tasks such as test-case generation, bug analysis, and traceability. Yet their industrial value remains unclear, partly because practicing testers’ perspectives are underexplored. Objective. We investigate how industry testers perceive the usefulness and limitations of LLMs across the Software Test Life Cycle (STLC), identifying (i) the most critical stages, (ii) operational bottlenecks, and (iii) perceived benefits, risks, and adoption conditions. Method. A focus group with five professional quality analysts (2–10 years’ experience) included hands-on interaction with an LLM-powered BDD notebook, followed by guided discussion and affinity-diagram voting. Forty-two statements were coded and prioritized. Results. Requirements Analysis and Test Planning were deemed most critical; 71% of votes linked rework to unclear or incomplete requirements. Benefits cited were automation of repetitive tasks, broader coverage, and faster learning; main concerns were prompt sensitivity, limited domain generalization, and data-privacy risks, with emphasis on human oversight and domain-adapted prompt libraries. Conclusion. While LLMs can improve efficiency and coverage, adoption depends on high-quality inputs, secure deployment, and early-phase integration. The findings offer empirically grounded guidance for aligning LLM solutions with the socio-technical realities of industrial testing teams.
- 23 September 2025
- Published by Sociedade Brasileira de Computação
- p. 25-30
Open AccessHow Software Testers Perceive Large Language Models: A Focus Group Study with Lexicometric Analysis
Context. Large Language Models (LLMs) are increasingly embedded in software-engineering toolchains, with potential to enhance quality-assurance tasks such as test-case generation, bug analysis, and traceability. Yet their industrial value remains unclear, partly because practicing testers’ perspectives are underexplored. Objective. We investigate how industry testers perceive the usefulness and limitations of LLMs across the Software Test Life Cycle (STLC), identifying (i) the most critical stages, (ii) operational bottlenecks, and (iii) perceived benefits, risks, and adoption conditions. Method. A focus group with five professional quality analysts (2–10 years’ experience) included hands-on interaction with an LLM-powered BDD notebook, followed by guided discussion and affinity-diagram voting. Forty-two statements were coded and prioritized. Results. Requirements Analysis and Test Planning were deemed most critical; 71% of votes linked rework to unclear or incomplete requirements. Benefits cited were automation of repetitive tasks, broader coverage, and faster learning; main concerns were prompt sensitivity, limited domain generalization, and data-privacy risks, with emphasis on human oversight and domain-adapted prompt libraries. Conclusion. While LLMs can improve efficiency and coverage, adoption depends on high-quality inputs, secure deployment, and early-phase integration. The findings offer empirically grounded guidance for aligning LLM solutions with the socio-technical realities of industrial testing teams.
- Open Access
- 22 September 2025
- Published by Sociedade Brasileira de Computação
- p. 171-181
A Comparative Study of LLMs for Gherkin Generation
[Context] Behavior-Driven Development (BDD) is widely adopted, but the manual creation of Gherkin scenarios remains a significant bottleneck. While Large Language Models (LLMs) show promise for automation, there is a lack of empirical evidence on their accuracy and stability when converting free-form test descriptions into structured Gherkin, creating risks for industrial adoption. Manual scenario authoring is also time-consuming and prone to inconsistencies, leading to miscommunication between technical and non-technical stakeholders and impacting software quality assurance. [Objective] This study addresses this gap by investigating the use of LLMs to automate the generation of Gherkin-based BDD scenarios from real-world, free-form test case descriptions. The goal is to assess the robustness of current models when handling informal, ambiguous, and diverse inputs typically found in practice. [Method]We conducted a comparative evaluation involving seven LLMs — GPT-3.5 Turbo, GPT-4 Turbo, GPT-4o Mini, LLaMA 3, Phi-3, Gemini, and DeepSeek R1 — using zero-shot, one-shot, and fewshot prompting strategies. The models generated BDD scenarios from a stratified sample of ten test descriptions selected from a corpus of 1,286, ensuring diversity in structure and domain complexity. We assessed quality and consistency using quantitative metrics (METEOR, variability analysis) and Repeated Measures ANOVA to test statistical significance. [Results] The analysis revealed that simple zero-shot prompting was highly effective, achieving results comparable to more complex example-based prompting. For the topperforming model, Gemini, which balanced accuracy and stability, the difference between zero-shot and few-shot was not statistically significant. Performance differences across models were often small, suggesting that practical factors like integration and cost should also guide model choice. Some models showed higher output variability, raising concerns about consistency in test generation workflows. [Conclusion] This paper offers practical insights into prompt design and model selection for LLM-based BDD scenario generation. Results show that effective zero-shot prompts can enable scalable, high-quality generation comparable to more complex techniques, simplifying LLM adoption in industrial testing. These findings suggest that LLMs can be leveraged with minimal setup to streamline BDD, reduce costs, and accelerate validation cycles.
- 22 September 2025
- Published by Sociedade Brasileira de Computação
- p. 171-181
Open AccessA Comparative Study of LLMs for Gherkin Generation
[Context] Behavior-Driven Development (BDD) is widely adopted, but the manual creation of Gherkin scenarios remains a significant bottleneck. While Large Language Models (LLMs) show promise for automation, there is a lack of empirical evidence on their accuracy and stability when converting free-form test descriptions into structured Gherkin, creating risks for industrial adoption. Manual scenario authoring is also time-consuming and prone to inconsistencies, leading to miscommunication between technical and non-technical stakeholders and impacting software quality assurance. [Objective] This study addresses this gap by investigating the use of LLMs to automate the generation of Gherkin-based BDD scenarios from real-world, free-form test case descriptions. The goal is to assess the robustness of current models when handling informal, ambiguous, and diverse inputs typically found in practice. [Method]We conducted a comparative evaluation involving seven LLMs — GPT-3.5 Turbo, GPT-4 Turbo, GPT-4o Mini, LLaMA 3, Phi-3, Gemini, and DeepSeek R1 — using zero-shot, one-shot, and fewshot prompting strategies. The models generated BDD scenarios from a stratified sample of ten test descriptions selected from a corpus of 1,286, ensuring diversity in structure and domain complexity. We assessed quality and consistency using quantitative metrics (METEOR, variability analysis) and Repeated Measures ANOVA to test statistical significance. [Results] The analysis revealed that simple zero-shot prompting was highly effective, achieving results comparable to more complex example-based prompting. For the topperforming model, Gemini, which balanced accuracy and stability, the difference between zero-shot and few-shot was not statistically significant. Performance differences across models were often small, suggesting that practical factors like integration and cost should also guide model choice. Some models showed higher output variability, raising concerns about consistency in test generation workflows. [Conclusion] This paper offers practical insights into prompt design and model selection for LLM-based BDD scenario generation. Results show that effective zero-shot prompts can enable scalable, high-quality generation comparable to more complex techniques, simplifying LLM adoption in industrial testing. These findings suggest that LLMs can be leveraged with minimal setup to streamline BDD, reduce costs, and accelerate validation cycles.