Federico Formica
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Research Interests
Federico Formica has a total of 16 co-authors. Here are the top co-authors by number of shared publications:
- Claudio Menghi
University of Bergamo, BERGAMO, Italy
6 shared publications - Mark Lawford
McMaster University
2 shared publications - Tony Fan
McMaster University, Hamilton, ON, Canada
2 shared publications - Vera Pantelic
McMaster University, McMaster Centre for Software Certification (McSCert), Hamilton, Ontario, Canada
2 shared publications - Akshay Rajhans
Mathworks, Natick, MA, USA
1 shared publication - Alessio Di Sandro
Department of Computer Science, University of Toronto, Toronto, Canada
1 shared publication
Latest Publications
- Open Access
- 7 October 2026
- Published by Association for Computing Machinery (ACM)
- p. 3920-3932
Test Suite Generation for Simulink Models: An Experience with Reactis
Testing Simulink models is a widely recognized, relevant, and challenging problem for automated software engineering. It is relevant to industry practitioners since Simulink is a development and simulation language widely used by the Cyber-Physical System (CPS) industry. It is challenging since CPS are usually complex, entail many components, and failures are not easily identifiable without automated support. This paper reports our experience in applying Reactis, an industrial tool for testing Simulink models, to the ten Cyber-Physical Challenges Simulink models from Lockheed Martin. We considered 48 requirements from these challenge models. We assess the effectiveness of Reactis in identifying failure-revealing test cases and its efficiency. We compare our results and those from an academic testing tool (HC), and two model checking tools (QVtrace and CoCoSim). Our results show that Reactis could return a failure-revealing test case for a requirement that was considered correct. They also show that the efficiency of Reactis is comparable to that of existing tools.
- 7 October 2026
- Published by Association for Computing Machinery (ACM)
- p. 3920-3932
Open AccessTest Suite Generation for Simulink Models: An Experience with Reactis
Testing Simulink models is a widely recognized, relevant, and challenging problem for automated software engineering. It is relevant to industry practitioners since Simulink is a development and simulation language widely used by the Cyber-Physical System (CPS) industry. It is challenging since CPS are usually complex, entail many components, and failures are not easily identifiable without automated support. This paper reports our experience in applying Reactis, an industrial tool for testing Simulink models, to the ten Cyber-Physical Challenges Simulink models from Lockheed Martin. We considered 48 requirements from these challenge models. We assess the effectiveness of Reactis in identifying failure-revealing test cases and its efficiency. We compare our results and those from an academic testing tool (HC), and two model checking tools (QVtrace and CoCoSim). Our results show that Reactis could return a failure-revealing test case for a requirement that was considered correct. They also show that the efficiency of Reactis is comparable to that of existing tools.
- Open Access
- 30 June 2026
- Published by Association for Computing Machinery (ACM) in Proceedings of the ACM on Software Engineering
- Vol. 3 (FSE) , 4575-4598
Automated Repair of Requirements for Cyber-Physical Systems in Simulink Requirements Tables
The development of complex software systems, e.g., cyber-physical systems (CPSs), involves continuous evolution of both system implementations and their requirements. These two artifacts often proceed independently, creating a risk of misalignment. For example, a system may be updated due to implementation-level concerns, yielding a new version that no longer satisfies its original requirements. Traditional compliance recovery techniques, e.g., automated program repair, address this problem by modifying the system while assuming that requirements are correct. However, faulty, outdated or inadequate requirements are a well-documented challenge in practice, motivating the complementary task of requirement repair. In this paper, we propose a framework that leverages system execution data to repair misaligned CPS requirements, thereby restoring requirement-to-system compliance. Our approach evaluates the correctness of declarative requirements over time-based, real-valued signals expressed using the MATLAB Simulink Requirements Tables language. We evaluate seven variants of our framework on six real-world case studies covering 12 requirements. Results confirm the effectiveness of the proposed framework in producing correct and useful repaired requirements.
- 30 June 2026
- Published by Association for Computing Machinery (ACM) in Proceedings of the ACM on Software Engineering
- Vol. 3 (FSE) , 4575-4598
Open AccessAutomated Repair of Requirements for Cyber-Physical Systems in Simulink Requirements Tables
The development of complex software systems, e.g., cyber-physical systems (CPSs), involves continuous evolution of both system implementations and their requirements. These two artifacts often proceed independently, creating a risk of misalignment. For example, a system may be updated due to implementation-level concerns, yielding a new version that no longer satisfies its original requirements. Traditional compliance recovery techniques, e.g., automated program repair, address this problem by modifying the system while assuming that requirements are correct. However, faulty, outdated or inadequate requirements are a well-documented challenge in practice, motivating the complementary task of requirement repair. In this paper, we propose a framework that leverages system execution data to repair misaligned CPS requirements, thereby restoring requirement-to-system compliance. Our approach evaluates the correctness of declarative requirements over time-based, real-valued signals expressed using the MATLAB Simulink Requirements Tables language. We evaluate seven variants of our framework on six real-world case studies covering 12 requirements. Results confirm the effectiveness of the proposed framework in producing correct and useful repaired requirements.
- Open Access
- 10 July 2024
- Published by Association for Computing Machinery (ACM)
- p. 587-591
ATheNA-S: A Testing Tool for Simulink Models Driven by Software Requirements and Domain Expertise
Search-based software testing (SBST) is widely used to verify software systems. SBST iteratively generates new test inputs driven by fitness functions, i.e., objective functions that guide the test case generation. In previous work, we proposed ATheNA, a novel SBST framework that combines fitness functions automatically generated from requirements' specifications with those manually defined by engineers, and showed its effectiveness. This tool demonstration paper describes ATheNA-S, an instance of ATheNA that targets Simulink models. We demonstrate our tool using an automotive case study and present our implementation and design decisions. A video walkthrough of the case study is available on YouTube: youtu.be/dhw9rwO7L4k.
- 10 July 2024
- Published by Association for Computing Machinery (ACM)
- p. 587-591
Open AccessATheNA-S: A Testing Tool for Simulink Models Driven by Software Requirements and Domain Expertise
Search-based software testing (SBST) is widely used to verify software systems. SBST iteratively generates new test inputs driven by fitness functions, i.e., objective functions that guide the test case generation. In previous work, we proposed ATheNA, a novel SBST framework that combines fitness functions automatically generated from requirements' specifications with those manually defined by engineers, and showed its effectiveness. This tool demonstration paper describes ATheNA-S, an instance of ATheNA that targets Simulink models. We demonstrate our tool using an automotive case study and present our implementation and design decisions. A video walkthrough of the case study is available on YouTube: youtu.be/dhw9rwO7L4k.
- 25 December 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Software Engineering
- Vol. 50 (2) , 239-257
Simulation-Based Testing of Simulink Models With Test Sequence and Test Assessment Blocks
Simulation-based software testing supports engineers in finding faults in Simulink® models. It typically relies on search algorithms that iteratively generate test inputs used to exercise models in simulation to detect design errors. While simulation-based software testing techniques are effective in many practical scenarios, they are typically not fully integrated within the Simulink environment and require additional manual effort. Many techniques require engineers to specify requirements using logical languages that are neither intuitive nor fully supported by Simulink, thereby limiting their adoption in industry. This work presents HECATE, a testing approach for Simulink models using Test Sequence and Test Assessment blocks from Simulink® TestTM. Unlike existing testing techniques, HECATE uses information from Simulink models to guide the search-based exploration. Specifically, HECATE relies on information provided by the Test Sequence and Test Assessment blocks to guide the search procedure. Across a benchmark of 18 Simulink models from different domains and industries, our comparison of HECATE with the state-of-the-art testing tool S-Taliro indicates that HECATE is both more effective (more failure-revealing test cases) and efficient (less iterations and computational time) than S-Taliro for ≈94% and ≈83% of benchmark models respectively. Furthermore, HECATE successfully generated a failure-revealing test case for a representative case study from the automotive domain demonstrating its practical usefulness.
- 25 December 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Software Engineering
- Vol. 50 (2) , 239-257
Simulation-Based Testing of Simulink Models With Test Sequence and Test Assessment Blocks
Simulation-based software testing supports engineers in finding faults in Simulink® models. It typically relies on search algorithms that iteratively generate test inputs used to exercise models in simulation to detect design errors. While simulation-based software testing techniques are effective in many practical scenarios, they are typically not fully integrated within the Simulink environment and require additional manual effort. Many techniques require engineers to specify requirements using logical languages that are neither intuitive nor fully supported by Simulink, thereby limiting their adoption in industry. This work presents HECATE, a testing approach for Simulink models using Test Sequence and Test Assessment blocks from Simulink® TestTM. Unlike existing testing techniques, HECATE uses information from Simulink models to guide the search-based exploration. Specifically, HECATE relies on information provided by the Test Sequence and Test Assessment blocks to guide the search procedure. Across a benchmark of 18 Simulink models from different domains and industries, our comparison of HECATE with the state-of-the-art testing tool S-Taliro indicates that HECATE is both more effective (more failure-revealing test cases) and efficient (less iterations and computational time) than S-Taliro for ≈94% and ≈83% of benchmark models respectively. Furthermore, HECATE successfully generated a failure-revealing test case for a representative case study from the automotive domain demonstrating its practical usefulness.
- Open Access
- 23 December 2023
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Software Engineering and Methodology
- Vol. 33 (2) , 1-37
Search-Based Software Testing Driven by Automatically Generated and Manually Defined Fitness Functions
Search-based software testing (SBST) typically relies on fitness functions to guide the search exploration toward software failures. There are two main techniques to define fitness functions: (a) automated fitness function computation from the specification of the system requirements, and (b) manual fitness function design. Both techniques have advantages. The former uses information from the system requirements to guide the search toward portions of the input domain more likely to contain failures. The latter uses the engineers’ domain knowledge. We propose ATheNA , a novel SBST framework that combines fitness functions automatically generated from requirements specifications and those manually defined by engineers. We design and implement ATheNA-S , an instance of ATheNA that targets Simulink ® models. We evaluate ATheNA-S by considering a large set of models from different domains. Our results show that ATheNA-S generates more failure-revealing test cases than existing baseline tools and that the difference between the runtime performance of ATheNA-S and the baseline tools is not statistically significant. We also assess whether ATheNA-S could generate failure-revealing test cases when applied to two representative case studies: one from the automotive domain and one from the medical domain. Our results show that ATheNA-S successfully revealed a requirement violation in our case studies.
- 23 December 2023
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Software Engineering and Methodology
- Vol. 33 (2) , 1-37
Open AccessSearch-Based Software Testing Driven by Automatically Generated and Manually Defined Fitness Functions
Search-based software testing (SBST) typically relies on fitness functions to guide the search exploration toward software failures. There are two main techniques to define fitness functions: (a) automated fitness function computation from the specification of the system requirements, and (b) manual fitness function design. Both techniques have advantages. The former uses information from the system requirements to guide the search toward portions of the input domain more likely to contain failures. The latter uses the engineers’ domain knowledge. We propose ATheNA , a novel SBST framework that combines fitness functions automatically generated from requirements specifications and those manually defined by engineers. We design and implement ATheNA-S , an instance of ATheNA that targets Simulink ® models. We evaluate ATheNA-S by considering a large set of models from different domains. Our results show that ATheNA-S generates more failure-revealing test cases than existing baseline tools and that the difference between the runtime performance of ATheNA-S and the baseline tools is not statistically significant. We also assess whether ATheNA-S could generate failure-revealing test cases when applied to two representative case studies: one from the automotive domain and one from the medical domain. Our results show that ATheNA-S successfully revealed a requirement violation in our case studies.
- Open Access
- 30 November 2023
- Published by Association for Computing Machinery (ACM)
- p. 1949-1960
Test Case Generation for Drivability Requirements of an Automotive Cruise Controller: An Experience with an Industrial Simulator
- 30 November 2023
- Published by Association for Computing Machinery (ACM)
- p. 1949-1960
Open AccessTest Case Generation for Drivability Requirements of an Automotive Cruise Controller: An Experience with an Industrial Simulator