Santiago Matalonga
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Research Subjects
Santiago Matalonga has a total of 61 co-authors. Here are the top co-authors by number of shared publications:
- Martin Solari
Universidad ORT Uruguay, Montevideo, Uruguay
6 shared publications - Domenico Amalfitano
Università di Napoli Federico II
5 shared publications Guilherme Horta Travassos
4 shared publications- Guilherme Horta Travassos
Federal University of Rio de Janeiro, Brazil
4 shared publications - Jean Carlo Rossa Hauck
UFSC
4 shared publications Guilherme H. Travassos
Universidade Federal do Rio de Janeiro, PESC/COPPE, Rio de Janeiro, Brazil
3 shared publications
Latest Publications
- Open Access
Accelerating Discovery: Rapid Literature Screening with LLMs
Background: Conducting Multi Vocal Literature Reviews (MVLRs) is often time and effort-intensive. Researchers must review and filter a large number of unstructured sources, which frequently contain sparse information and are unlikely to be included in the final study. Our experience conducting an MVLR on Context-Aware Software Systems (CASS) Testing in the avionics domain exemplified this challenge, with over 8,000 highly heterogeneous documents requiring review. Therefore, we developed a Large Language Model (LLM) assistant to support the search and filtering of documents. Aims: To develop and validate an LLM based tool that can support researchers in performing the search and filtering of documents for an MVLR without compromising the rigor of the research protocol. Method: We applied sound engineering practices to develop an on-premises LLM-based tool incorporating Retrieval Augmented Generation (RAG) to process candidate sources. Progress towards the aim was quantified using the Positive Percent Agreement (PPA) as the primary metric to ensure the performance of the LLM based tool. Convenience sampling, supported by human judgment and statistical sampling, were used to verify and validate the tool's quality-in-use. Results: The tool currently demonstrates a PPA agreement with human researchers of 90% for sources that are not relevant to the study. Development details are shared to support domain-specific adaptation of the tool. Conclusions: Using LLM-based tools to support academic researchers in rigorous MVLR is feasible. These tools can free valuable time for higher-level, abstract tasks. However, researcher participation remains essential to ensure that the tool supports thorough research.
Open AccessAccelerating Discovery: Rapid Literature Screening with LLMs
Background: Conducting Multi Vocal Literature Reviews (MVLRs) is often time and effort-intensive. Researchers must review and filter a large number of unstructured sources, which frequently contain sparse information and are unlikely to be included in the final study. Our experience conducting an MVLR on Context-Aware Software Systems (CASS) Testing in the avionics domain exemplified this challenge, with over 8,000 highly heterogeneous documents requiring review. Therefore, we developed a Large Language Model (LLM) assistant to support the search and filtering of documents. Aims: To develop and validate an LLM based tool that can support researchers in performing the search and filtering of documents for an MVLR without compromising the rigor of the research protocol. Method: We applied sound engineering practices to develop an on-premises LLM-based tool incorporating Retrieval Augmented Generation (RAG) to process candidate sources. Progress towards the aim was quantified using the Positive Percent Agreement (PPA) as the primary metric to ensure the performance of the LLM based tool. Convenience sampling, supported by human judgment and statistical sampling, were used to verify and validate the tool's quality-in-use. Results: The tool currently demonstrates a PPA agreement with human researchers of 90% for sources that are not relevant to the study. Development details are shared to support domain-specific adaptation of the tool. Conclusions: Using LLM-based tools to support academic researchers in rigorous MVLR is feasible. These tools can free valuable time for higher-level, abstract tasks. However, researcher participation remains essential to ensure that the tool supports thorough research.
- 30 June 2025
- Published by Springer Nature in Software Quality Journal
- Vol. 33 (3) , 1-31
A study on reported under-documented non-functional requirements as an indicator of technical debt
- 30 June 2025
- Published by Springer Nature in Software Quality Journal
- Vol. 33 (3) , 1-31
A study on reported under-documented non-functional requirements as an indicator of technical debt
- 7 February 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Industrial Informatics
- Vol. 21 (5) , 3705-3716
Testing Context-Aware Software Systems From the Voices of the Automotive Industry
As automotive software systems evolve toward high and full driving automation, evaluating their quality becomes increasingly challenging, especially concerning emerging behaviors. Context awareness is the capability to sense the environment and adapt behavior. Automotive software systems are context-aware software systems (CASS). Previous secondary studies in technical literature indicate a need for testing techniques for CASS. However, these studies should have investigated the information provided by the industry. Therefore, this article undertakes a gray literature study to uncover evidence of CASS testing using 20 reports from 16 automotive companies as primary sources. Our findings show that industry practices exhibit quality assurance best practices, but CASS abstraction adoption still needs to be completed. Industry reports emphasize testing challenges but lack technical resolutions, relying on amassing diverse datasets for testing. This article has the potential to impact the quality assurance of automotive software systems significantly and lead industry professionals to enhance their testing process.
- 7 February 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Industrial Informatics
- Vol. 21 (5) , 3705-3716
Testing Context-Aware Software Systems From the Voices of the Automotive Industry
As automotive software systems evolve toward high and full driving automation, evaluating their quality becomes increasingly challenging, especially concerning emerging behaviors. Context awareness is the capability to sense the environment and adapt behavior. Automotive software systems are context-aware software systems (CASS). Previous secondary studies in technical literature indicate a need for testing techniques for CASS. However, these studies should have investigated the information provided by the industry. Therefore, this article undertakes a gray literature study to uncover evidence of CASS testing using 20 reports from 16 automotive companies as primary sources. Our findings show that industry practices exhibit quality assurance best practices, but CASS abstraction adoption still needs to be completed. Industry reports emphasize testing challenges but lack technical resolutions, relying on amassing diverse datasets for testing. This article has the potential to impact the quality assurance of automotive software systems significantly and lead industry professionals to enhance their testing process.
- 30 August 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Intelligent Transportation Systems
- Vol. 25 (11) , 17762-17777
Segmentation of Drone Collision Hazards in Airborne RADAR Point Clouds Using PointNet
The integration of drones into shared airspace for beyond visual line of sight (BVLOS) operations presents significant challenges but holds transformative potential for sectors like transportation, construction, energy and defence. A prerequisite for this integration is equipping drones with enhanced situational awareness to ensure collision avoidance and safe operations. Current approaches mainly target single object detection or classification, or simpler sensing outputs that offer limited perceptual understanding and lack the rapid end-to-end processing needed to convert sensor data into safety-critical insights. In contrast, our study leverages radar technology for novel end-to-end semantic segmentation of aerial point clouds to simultaneously identify multiple collision hazards. By adapting and optimizing the PointNet architecture and integrating aerial domain insights, our framework distinguishes five distinct classes: mobile targets like drones (DJI M300 and DJI Mini) and airplanes (Ikarus C42), and static returns (ground and infrastructure) which results in enhanced situational awareness for drones. To our knowledge, this is the first approach addressing simultaneous identification of multiple collision threats in an aerial setting, achieving a robust 94% accuracy. This work highlights the potential of radar technology to advance situational awareness in drones, facilitating safe and efficient BVLOS operations.
- 30 August 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Intelligent Transportation Systems
- Vol. 25 (11) , 17762-17777
Segmentation of Drone Collision Hazards in Airborne RADAR Point Clouds Using PointNet
The integration of drones into shared airspace for beyond visual line of sight (BVLOS) operations presents significant challenges but holds transformative potential for sectors like transportation, construction, energy and defence. A prerequisite for this integration is equipping drones with enhanced situational awareness to ensure collision avoidance and safe operations. Current approaches mainly target single object detection or classification, or simpler sensing outputs that offer limited perceptual understanding and lack the rapid end-to-end processing needed to convert sensor data into safety-critical insights. In contrast, our study leverages radar technology for novel end-to-end semantic segmentation of aerial point clouds to simultaneously identify multiple collision hazards. By adapting and optimizing the PointNet architecture and integrating aerial domain insights, our framework distinguishes five distinct classes: mobile targets like drones (DJI M300 and DJI Mini) and airplanes (Ikarus C42), and static returns (ground and infrastructure) which results in enhanced situational awareness for drones. To our knowledge, this is the first approach addressing simultaneous identification of multiple collision threats in an aerial setting, achieving a robust 94% accuracy. This work highlights the potential of radar technology to advance situational awareness in drones, facilitating safe and efficient BVLOS operations.
- Open Access
- 24 July 2024
- Published by Springer Nature in Journal of Intelligent & Robotic Systems
- Vol. 110 (3) , 1-18
Verification and Validation for a Digital Twin for Augmenting Current SORA Practices with Air-to-Air Collision Hazards Prediction from Small Uncooperative Flying Objects
Future autonomous Unmanned Aerial Vehicles (UAV) missions will take place in highly cluttered urban environments. As a result, the UAV must be able to autonomously evaluate risks and react to unforeseen hazards. The current regulatory framework for missions implements SORA guidelines for hazard detection, but its application to air-to-air collision is limited. This research defined a rigorous verification and validation framework (V&V) for digital twins for use in future autonomous UAV missions. The researchers designed a sentry mission for a UAV to evaluate its capacity to detect small uncooperative flying objects. A digital twin of the DJI M300 vision system was built using a game engine and a V&V framework was developed to assure the quality of results in both virtual and real-world scenarios. The results showed the capability of the digital twin to identify vulnerabilities and worst-case scenarios in UAV mission operations, and how it can assist remote pilots in identifying air-to-air collision hazards. Furthermore, the probability of air-to-air collision was calculated for three sentry patterns, and the results were validated in the field. This research demonstrated the capability to identify vulnerabilities and worst-case scenarios in UAV mission operations. We present how the digital twin of an operational theatre can be exploited to assist remote pilots with the identification of air-to-air collision hazards of small uncooperative objects. Furthermore, we discuss how these results can be used to enhance current SORA-based risk assessment practices.
- 24 July 2024
- Published by Springer Nature in Journal of Intelligent & Robotic Systems
- Vol. 110 (3) , 1-18
Open AccessVerification and Validation for a Digital Twin for Augmenting Current SORA Practices with Air-to-Air Collision Hazards Prediction from Small Uncooperative Flying Objects
Future autonomous Unmanned Aerial Vehicles (UAV) missions will take place in highly cluttered urban environments. As a result, the UAV must be able to autonomously evaluate risks and react to unforeseen hazards. The current regulatory framework for missions implements SORA guidelines for hazard detection, but its application to air-to-air collision is limited. This research defined a rigorous verification and validation framework (V&V) for digital twins for use in future autonomous UAV missions. The researchers designed a sentry mission for a UAV to evaluate its capacity to detect small uncooperative flying objects. A digital twin of the DJI M300 vision system was built using a game engine and a V&V framework was developed to assure the quality of results in both virtual and real-world scenarios. The results showed the capability of the digital twin to identify vulnerabilities and worst-case scenarios in UAV mission operations, and how it can assist remote pilots in identifying air-to-air collision hazards. Furthermore, the probability of air-to-air collision was calculated for three sentry patterns, and the results were validated in the field. This research demonstrated the capability to identify vulnerabilities and worst-case scenarios in UAV mission operations. We present how the digital twin of an operational theatre can be exploited to assist remote pilots with the identification of air-to-air collision hazards of small uncooperative objects. Furthermore, we discuss how these results can be used to enhance current SORA-based risk assessment practices.
- Open Access
- 6 October 2023
- Published by Association for Computing Machinery (ACM) in ACM Computing Surveys
- Vol. 56 (3) , 1-38
Artificial Intelligence Applied to Software Testing: A Tertiary Study
Context: Artificial intelligence (AI) methods and models have extensively been applied to support different phases of the software development lifecycle, including software testing (ST). Several secondary studies investigated the interplay between AI and ST but restricted the scope of the research to specific domains or sub-domains within either area. Objective: This research aims to explore the overall contribution of AI to ST, while identifying the most popular applications and potential paths for future research directions. Method: We executed a tertiary study following well-established guidelines for conducting systematic literature mappings in software engineering and for answering nine research questions. Results : We identified and analyzed 20 relevant secondary studies. The analysis was performed by drawing from well-recognized AI and ST taxonomies and mapping the selected studies according to them. The resulting mapping and discussions provide extensive and detailed information on the interplay between AI and ST. Conclusion: The application of AI to support ST is a well-consolidated and growing interest research topic. The mapping resulting from our study can be used by researchers to identify opportunities for future research, and by practitioners looking for evidence-based information on which AI-supported technology to possibly adopt in their testing processes.
- 6 October 2023
- Published by Association for Computing Machinery (ACM) in ACM Computing Surveys
- Vol. 56 (3) , 1-38
Open AccessArtificial Intelligence Applied to Software Testing: A Tertiary Study
Context: Artificial intelligence (AI) methods and models have extensively been applied to support different phases of the software development lifecycle, including software testing (ST). Several secondary studies investigated the interplay between AI and ST but restricted the scope of the research to specific domains or sub-domains within either area. Objective: This research aims to explore the overall contribution of AI to ST, while identifying the most popular applications and potential paths for future research directions. Method: We executed a tertiary study following well-established guidelines for conducting systematic literature mappings in software engineering and for answering nine research questions. Results : We identified and analyzed 20 relevant secondary studies. The analysis was performed by drawing from well-recognized AI and ST taxonomies and mapping the selected studies according to them. The resulting mapping and discussions provide extensive and detailed information on the interplay between AI and ST. Conclusion: The application of AI to support ST is a well-consolidated and growing interest research topic. The mapping resulting from our study can be used by researchers to identify opportunities for future research, and by practitioners looking for evidence-based information on which AI-supported technology to possibly adopt in their testing processes.
- Top Cited PapersOpen Access
- 22 August 2022
- Published by Springer Nature in Journal of Intelligent & Robotic Systems
- Vol. 106 (1) , 1-13
A Review of the Legal, Regulatory and Practical Aspects Needed to Unlock Autonomous Beyond Visual Line of Sight Unmanned Aircraft Systems Operations
Services that exploit Unmanned Aircraft Systems (UAS) are poised to revolutionise the service industry with a projected value of 71 BUSD by the end of the decade. A key enabler of this technology is the unlocking of autonomous Beyond Visual Line of Sight (BVLOS) operations. BVLOS operations will depend on a robust Detect and Avoid (D&A) capability. Yet, currently in the UK and EU, BVLOS operations are only allowed in specific cases and scenarios. As a result, the technological landscape for the development of robust D&A faces limitations, and there is little market incentive for development. Furthermore, while automated BVLOS is a future technology, a strong D&A capability is of importance now for all types of UAS operations. As the remote pilot has to deal with information overload from the controller device and the environment. These high-risk UAS operations are becoming more common. In this paper, we discuss the current legal framework in the UK making comparisons to EU countries. We make the case that even when an operation abides by the current framework the remote pilot is exposed to several legal liabilities. We review the roadmaps for UAS adoption (including certification processes for UAS-based products) and highlight that for software-intensive systems, key steps are missing to assure the quality of the product. Finally, we build on these findings to set forwards a path to complement future certification processes to enable autonomous based UAS operations to share the airspace with remotely piloted operations.
- 22 August 2022
- Published by Springer Nature in Journal of Intelligent & Robotic Systems
- Vol. 106 (1) , 1-13
Top Cited PapersOpen AccessA Review of the Legal, Regulatory and Practical Aspects Needed to Unlock Autonomous Beyond Visual Line of Sight Unmanned Aircraft Systems Operations
Services that exploit Unmanned Aircraft Systems (UAS) are poised to revolutionise the service industry with a projected value of 71 BUSD by the end of the decade. A key enabler of this technology is the unlocking of autonomous Beyond Visual Line of Sight (BVLOS) operations. BVLOS operations will depend on a robust Detect and Avoid (D&A) capability. Yet, currently in the UK and EU, BVLOS operations are only allowed in specific cases and scenarios. As a result, the technological landscape for the development of robust D&A faces limitations, and there is little market incentive for development. Furthermore, while automated BVLOS is a future technology, a strong D&A capability is of importance now for all types of UAS operations. As the remote pilot has to deal with information overload from the controller device and the environment. These high-risk UAS operations are becoming more common. In this paper, we discuss the current legal framework in the UK making comparisons to EU countries. We make the case that even when an operation abides by the current framework the remote pilot is exposed to several legal liabilities. We review the roadmaps for UAS adoption (including certification processes for UAS-based products) and highlight that for software-intensive systems, key steps are missing to assure the quality of the product. Finally, we build on these findings to set forwards a path to complement future certification processes to enable autonomous based UAS operations to share the airspace with remotely piloted operations.
- Open Access
- 6 May 2022
- Published by Elsevier in Information and Software Technology
- Vol. 149
Alternatives for testing of context-aware software systems in non-academic settings: results from a Rapid Review
- 6 May 2022
- Published by Elsevier in Information and Software Technology
- Vol. 149
Open AccessAlternatives for testing of context-aware software systems in non-academic settings: results from a Rapid Review
- 8 August 2021
- Published by Springer Nature
- p. 106-119
Applicability of the Software Security Code Metrics for Ethereum Smart Contract
- 8 August 2021
- Published by Springer Nature
- p. 106-119
Applicability of the Software Security Code Metrics for Ethereum Smart Contract
- Open Access
Alternatives for Testing of Context-Aware Contemporary Software Systems in industrial settings: Results from a Rapid review
Context: Context-aware contemporary software systems (CACSS) are mainstream. Furthermore, they present challenges for current engineering practices. These challenges are distinctively present when testing CACSS, as the variation of context deepens the limitations of available software testing practices and technologies. Objective: To understand how the industry deals with the variation of context when testing CACSS. Method: A Rapid Review was commissioned to uncover the necessary evidence to achieve the objectives. Results: Our results show that current research initiatives aim to generate or improve Test Suites that can deal with the variation of context and the sheer volume of test input possibilities. To achieve this, they mostly rely on modelling the systems' dynamic behavior and increasing computing resources to generate test inputs. We found no evidence of research results aiming at managing context variation through the testing lifecycle process. Conclusions: We discuss how the identified solutions are not ready for mainstream adoption. They are all domain-specific, and while the ideas and approaches can be reproduced in different settings, the technologies noon to be re-engineered and tailor to the specific CACSS.
Open AccessAlternatives for Testing of Context-Aware Contemporary Software Systems in industrial settings: Results from a Rapid review
Context: Context-aware contemporary software systems (CACSS) are mainstream. Furthermore, they present challenges for current engineering practices. These challenges are distinctively present when testing CACSS, as the variation of context deepens the limitations of available software testing practices and technologies. Objective: To understand how the industry deals with the variation of context when testing CACSS. Method: A Rapid Review was commissioned to uncover the necessary evidence to achieve the objectives. Results: Our results show that current research initiatives aim to generate or improve Test Suites that can deal with the variation of context and the sheer volume of test input possibilities. To achieve this, they mostly rely on modelling the systems' dynamic behavior and increasing computing resources to generate test inputs. We found no evidence of research results aiming at managing context variation through the testing lifecycle process. Conclusions: We discuss how the identified solutions are not ready for mainstream adoption. They are all domain-specific, and while the ideas and approaches can be reproduced in different settings, the technologies noon to be re-engineered and tailor to the specific CACSS.