Michael Foster
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Michael Foster has a total of 34 co-authors. Here are the top co-authors by number of shared publications:
- Neil Walkinshaw
The University of Sheffield, Sheffield, United Kingdom
8 shared publications - Robert Mark Hierons
University of Sheffield, Sheffield, UK
6 shared publications - Andrew G. Clark
University of Sheffield, UK
3 shared publications - John Derrick
University of Sheffield
3 shared publications - Ramsay Taylor
The University of Sheffield
3 shared publications - Robert Turner
Modernising Medical Microbiology Unit, Nuffield Department of Medicine, University of Oxford
3 shared publications
Latest Publications
- Open Access
- 7 October 2026
- Published by Association for Computing Machinery (ACM)
- p. 3586-3590
FlakeFighters: A Pytest Plugin to Detect Flaky Test Failures
Flaky tests — tests which non-deterministically pass and fail against the same version of code — pose a well-established painful issue for developers due to the unpredictability of failures. The current state of practice to detect and work around flaky tests is to simply rerun them. However, this can erode confidence in failing test outcomes and represents a significant waste of time and compute resources. A better solution is to detect flaky tests based on static data (collectable without running any tests) and/or execution data (collected by running the test suite) to determine if a test failure has occurred due to flaky behaviour or a genuine bug. Several such techniques have appeared in recent literature, but these exist in isolation as prototype research tools, and typically only support Java codebases. This paper presents FlakeFighters, an extensible pytest extension that implements a "Swiss army knife" of flaky test classification tools to help developers classify the underlying cause of failing tests as genuine bugs or flakiness in Python codebases. We ran pytest with the FlakeFighters extension on 23 failing workflows from the home-assistant/core repository and found that it was able to identify all of the failing flaky tests. A video demonstration of FlakeFighters can be found at https://youtu.be/3J4odtgzQX0.
- 7 October 2026
- Published by Association for Computing Machinery (ACM)
- p. 3586-3590
Open AccessFlakeFighters: A Pytest Plugin to Detect Flaky Test Failures
Flaky tests — tests which non-deterministically pass and fail against the same version of code — pose a well-established painful issue for developers due to the unpredictability of failures. The current state of practice to detect and work around flaky tests is to simply rerun them. However, this can erode confidence in failing test outcomes and represents a significant waste of time and compute resources. A better solution is to detect flaky tests based on static data (collectable without running any tests) and/or execution data (collected by running the test suite) to determine if a test failure has occurred due to flaky behaviour or a genuine bug. Several such techniques have appeared in recent literature, but these exist in isolation as prototype research tools, and typically only support Java codebases. This paper presents FlakeFighters, an extensible pytest extension that implements a "Swiss army knife" of flaky test classification tools to help developers classify the underlying cause of failing tests as genuine bugs or flakiness in Python codebases. We ran pytest with the FlakeFighters extension on 23 failing workflows from the home-assistant/core repository and found that it was able to identify all of the failing flaky tests. A video demonstration of FlakeFighters can be found at https://youtu.be/3J4odtgzQX0.
- Open Access
- 21 May 2026
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Software Engineering and Methodology
Causality-Driven Test Case Minimisation for Cyber-Physical Systems
Cyber-physical systems allow digital control systems to interact with the physical world using sensors and actuators. They are increasingly being used to automate critical infrastructure, where software faults can have dire consequences. Due to the complex nature and unpredictability of these systems, their resilience is often tested using a technique called fuzzing, which generates quasi-random sequences of sensor and actuator manipulations with the goal of forcing a system into unsafe states. However, there is currently no way of determining which manipulations of a test case cause a failure without systematically removing each one and re-running the test, which can be extremely time-consuming and expensive. In this work, we present CausalCut , a technique that uses causal inference to estimate the causal contribution of each intervention from pre-existing runtime data, thereby reducing the number of times tests must be re-run. We evaluated CausalCut by applying it to two very different systems: an artificial pancreas and a water treatment plant. CausalCut typically managed to remove more than half of the spurious manipulations using fewer executions than the current state of the art, which represents a saving of up to 18 hours and $6300 per test case.
- 21 May 2026
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Software Engineering and Methodology
Open AccessCausality-Driven Test Case Minimisation for Cyber-Physical Systems
Cyber-physical systems allow digital control systems to interact with the physical world using sensors and actuators. They are increasingly being used to automate critical infrastructure, where software faults can have dire consequences. Due to the complex nature and unpredictability of these systems, their resilience is often tested using a technique called fuzzing, which generates quasi-random sequences of sensor and actuator manipulations with the goal of forcing a system into unsafe states. However, there is currently no way of determining which manipulations of a test case cause a failure without systematically removing each one and re-running the test, which can be extremely time-consuming and expensive. In this work, we present CausalCut , a technique that uses causal inference to estimate the causal contribution of each intervention from pre-existing runtime data, thereby reducing the number of times tests must be re-run. We evaluated CausalCut by applying it to two very different systems: an artificial pancreas and a water treatment plant. CausalCut typically managed to remove more than half of the spurious manipulations using fewer executions than the current state of the art, which represents a saving of up to 18 hours and $6300 per test case.
- 30 April 2026
- Published by Springer Nature
- p. 381-417
Chatbot-Assisted Explainability of AI-Powered Models and Collection of User Feedback
- 30 April 2026
- Published by Springer Nature
- p. 381-417
Chatbot-Assisted Explainability of AI-Powered Models and Collection of User Feedback
- Open Access
- 23 June 2025
- Published by Association for Computing Machinery (ACM)
- p. 1394-1401
Evolving Estimation Models for Causal Testing
- 23 June 2025
- Published by Association for Computing Machinery (ACM)
- p. 1394-1401
Open AccessEvolving Estimation Models for Causal Testing
- Open Access
- 17 June 2025
- Published by Association for Computing Machinery (ACM)
- p. 592-603
Using Causal Inference to Test Systems with Hidden and Interacting Variables: An Evaluative Case Study
- 17 June 2025
- Published by Association for Computing Machinery (ACM)
- p. 592-603
Open AccessUsing Causal Inference to Test Systems with Hidden and Interacting Variables: An Evaluative Case Study
- Open Access
- 6 March 2025
- Published by The Open Journal in Journal of Open Source Software
- Vol. 10 (107) , 7739
The Causal Testing Framework
Scientific models possess several properties that make them notoriously difficult to test, including a complex input space, long execution times, and non-determinism, rendering existing testing techniques impractical. In fields such as epidemiology, where researchers seek answers to challenging causal questions, a statistical methodology known as Causal Inference (CI) (Hernán & Robins, 2020; Pearl, 2009) has addressed similar problems, enabling the inference of causal conclusions from noisy, biased, and sparse observational data instead of costly randomised trials. CI works by using domain knowledge to identify and mitigate for biases in the data, enabling them to answer causal questions that concern the effect of changing some feature on the observed outcome. The Causal Testing Framework (CTF) is a software testing framework that uses CI techniques to establish causal effects between software variables from pre-existing runtime data rather than having to collect bespoke, highly curated datasets especially for testing.
- 6 March 2025
- Published by The Open Journal in Journal of Open Source Software
- Vol. 10 (107) , 7739
Open AccessThe Causal Testing Framework
Scientific models possess several properties that make them notoriously difficult to test, including a complex input space, long execution times, and non-determinism, rendering existing testing techniques impractical. In fields such as epidemiology, where researchers seek answers to challenging causal questions, a statistical methodology known as Causal Inference (CI) (Hernán & Robins, 2020; Pearl, 2009) has addressed similar problems, enabling the inference of causal conclusions from noisy, biased, and sparse observational data instead of costly randomised trials. CI works by using domain knowledge to identify and mitigate for biases in the data, enabling them to answer causal questions that concern the effect of changing some feature on the observed outcome. The Causal Testing Framework (CTF) is a software testing framework that uses CI techniques to establish causal effects between software variables from pre-existing runtime data rather than having to collect bespoke, highly curated datasets especially for testing.
- Open Access
- 12 July 2024
- Published by Association for Computing Machinery (ACM) in Proceedings of the ACM on Software Engineering
- Vol. 1 (FSE) , 2538-2560
Bounding Random Test Set Size with Computational Learning Theory
Random testing approaches work by generating inputs at random, or by selecting inputs randomly from some pre-defined operational profile. One long-standing question that arises in this and other testing contexts is as follows: When can we stop testing? At what point can we be certain that executing further tests in this manner will not explore previously untested (and potentially buggy) software behaviors? This is analogous to the question in Machine Learning, of how many training examples are required in order to infer an accurate model. In this paper we show how probabilistic approaches to answer this question in Machine Learning (arising from Computational Learning Theory) can be applied in our testing context, to provide an upper-bound on the number of tests required to achieve a given level of adequacy. We validate this bound on a large set of Java units, and an autonomous driving system.
- 12 July 2024
- Published by Association for Computing Machinery (ACM) in Proceedings of the ACM on Software Engineering
- Vol. 1 (FSE) , 2538-2560
Open AccessBounding Random Test Set Size with Computational Learning Theory
Random testing approaches work by generating inputs at random, or by selecting inputs randomly from some pre-defined operational profile. One long-standing question that arises in this and other testing contexts is as follows: When can we stop testing? At what point can we be certain that executing further tests in this manner will not explore previously untested (and potentially buggy) software behaviors? This is analogous to the question in Machine Learning, of how many training examples are required in order to infer an accurate model. In this paper we show how probabilistic approaches to answer this question in Machine Learning (arising from Computational Learning Theory) can be applied in our testing context, to provide an upper-bound on the number of tests required to achieve a given level of adequacy. We validate this bound on a large set of Java units, and an autonomous driving system.
- Open Access
- 24 November 2023
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Software Engineering and Methodology
- Vol. 33 (1) , 1-42
Testing Causality in Scientific Modelling Software
From simulating galaxy formation to viral transmission in a pandemic, scientific models play a pivotal role in developing scientific theories and supporting government policy decisions that affect us all. Given these critical applications, a poor modelling assumption or bug could have far-reaching consequences. However, scientific models possess several properties that make them notoriously difficult to test, including a complex input space, long execution times, and non-determinism, rendering existing testing techniques impractical. In fields such as epidemiology, where researchers seek answers to challenging causal questions, a statistical methodology known as Causal inference has addressed similar problems, enabling the inference of causal conclusions from noisy, biased, and sparse data instead of costly experiments. This article introduces the causal testing framework: a framework that uses causal inference techniques to establish causal effects from existing data, enabling users to conduct software testing activities concerning the effect of a change, such as metamorphic testing, a posteriori . We present three case studies covering real-world scientific models, demonstrating how the causal testing framework can infer metamorphic test outcomes from reused, confounded test data to provide an efficient solution for testing scientific modelling software.
- 24 November 2023
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Software Engineering and Methodology
- Vol. 33 (1) , 1-42
Open AccessTesting Causality in Scientific Modelling Software
From simulating galaxy formation to viral transmission in a pandemic, scientific models play a pivotal role in developing scientific theories and supporting government policy decisions that affect us all. Given these critical applications, a poor modelling assumption or bug could have far-reaching consequences. However, scientific models possess several properties that make them notoriously difficult to test, including a complex input space, long execution times, and non-determinism, rendering existing testing techniques impractical. In fields such as epidemiology, where researchers seek answers to challenging causal questions, a statistical methodology known as Causal inference has addressed similar problems, enabling the inference of causal conclusions from noisy, biased, and sparse data instead of costly experiments. This article introduces the causal testing framework: a framework that uses causal inference techniques to establish causal effects from existing data, enabling users to conduct software testing activities concerning the effect of a change, such as metamorphic testing, a posteriori . We present three case studies covering real-world scientific models, demonstrating how the causal testing framework can infer metamorphic test outcomes from reused, confounded test data to provide an efficient solution for testing scientific modelling software.
- Open Access
- 7 July 2023
- Published by Royal College of Psychiatrists in BJPsych Open
- Vol. 9 (S1) , S21-S22
Targeted Approach to Providing Child and Adolescent Mental Health Education to GPs
Aims: The aim of this project was to construct and deliver an educational session for general practitioners (GPs) in local Primary Care Networks on challenging child and adolescent mental health conditions. It was hypothesized that delivering targeted teaching sessions, supported by the same quiz applied before and after, would demonstrate an effective and repeatable method of improving GPs’ knowledge about these conditions. Since the COVID-19 pandemic, demand on both Child and Adolescent Mental Health Services (CAMHS) and GPs has reached unprecedented levels. Compounding this load, half of all referrals written by GPs to CAMHS are rejected, which prolongs the time a young person is under GP care, delaying specialist intervention. Unfortunately, during GP training exposure to CAMHS is limited and dedicated teaching is often insufficient. As a step towards addressing this challenge, a comprehensive teaching session combining didactic and socratic methods was devised and tested.Methods: The teaching session comprised the presentation, diagnosis, and first steps in management of four challenging conditions in children: autism, eating disorders, depression, and emotional dysregulation. A quiz with multiple-choice answers was administered before and after the presentation, addressing each of the four conditions. The data collection took place between December 2022 and January 2023. A total of 29 pairs of quizzes were completed by GPs. Due to the type and size of data collected, a non-parametric bootstrap resampling method was used to compare the before-and-after scores for each topic and overall score.Results: For the 29 pairs of quizzes, mean differences and 95% confidence intervals (CIs) were calculated between before-and-after scores, for each topic and for the total. All 4 topics showed statistically significant mean improvements: autism 1.3 CI: [0.9 , 1.8 ], eating disorder 1.8 CI: [1.4 , 2.3 ], depression 1.4 CI: [1.0 , 1.7 ] and emotional dysregulation 1.7 CI: [1.4 , 2.0 ]. The total mean improvement was 6.2 CI: [5.5 , 6.8 ] out of a maximum 16 points.Conclusion: These targeted educational sessions suggest it is possible to make reliable improvements in GP knowledge across a variety of topics. With child and adolescent mental health demands at record levels, a more focused approach of the kind considered here may offer a model for training elsewhere. As an indication of the impact of this approach, further sessions on other topics have been requested by the GP teaching leads.
- 7 July 2023
- Published by Royal College of Psychiatrists in BJPsych Open
- Vol. 9 (S1) , S21-S22
Open AccessTargeted Approach to Providing Child and Adolescent Mental Health Education to GPs
Aims: The aim of this project was to construct and deliver an educational session for general practitioners (GPs) in local Primary Care Networks on challenging child and adolescent mental health conditions. It was hypothesized that delivering targeted teaching sessions, supported by the same quiz applied before and after, would demonstrate an effective and repeatable method of improving GPs’ knowledge about these conditions. Since the COVID-19 pandemic, demand on both Child and Adolescent Mental Health Services (CAMHS) and GPs has reached unprecedented levels. Compounding this load, half of all referrals written by GPs to CAMHS are rejected, which prolongs the time a young person is under GP care, delaying specialist intervention. Unfortunately, during GP training exposure to CAMHS is limited and dedicated teaching is often insufficient. As a step towards addressing this challenge, a comprehensive teaching session combining didactic and socratic methods was devised and tested.Methods: The teaching session comprised the presentation, diagnosis, and first steps in management of four challenging conditions in children: autism, eating disorders, depression, and emotional dysregulation. A quiz with multiple-choice answers was administered before and after the presentation, addressing each of the four conditions. The data collection took place between December 2022 and January 2023. A total of 29 pairs of quizzes were completed by GPs. Due to the type and size of data collected, a non-parametric bootstrap resampling method was used to compare the before-and-after scores for each topic and overall score.Results: For the 29 pairs of quizzes, mean differences and 95% confidence intervals (CIs) were calculated between before-and-after scores, for each topic and for the total. All 4 topics showed statistically significant mean improvements: autism 1.3 CI: [0.9 , 1.8 ], eating disorder 1.8 CI: [1.4 , 2.3 ], depression 1.4 CI: [1.0 , 1.7 ] and emotional dysregulation 1.7 CI: [1.4 , 2.0 ]. The total mean improvement was 6.2 CI: [5.5 , 6.8 ] out of a maximum 16 points.Conclusion: These targeted educational sessions suggest it is possible to make reliable improvements in GP knowledge across a variety of topics. With child and adolescent mental health demands at record levels, a more focused approach of the kind considered here may offer a model for training elsewhere. As an indication of the impact of this approach, further sessions on other topics have been requested by the GP teaching leads.