Testing Causality in Scientific Modelling Software
Preprint
Abstract
From simulating galaxy formation to viral transmission in a pandemic, scientific models play a pivotal role in developing scientific theories and supporting government policy decisions that affect us all. Given these critical applications, a poor modelling assumption or bug could have far-reaching consequences. However, scientific models possess several properties that make them notoriously difficult to test, including a complex input space, long execution times, and non-determinism, rendering existing testing techniques impractical. In fields such as epidemiology, where researchers seek answers to challenging causal questions, a statistical methodology known as Causal Inference has addressed similar problems, enabling the inference of causal conclusions from noisy, biased, and sparse data instead of costly experiments. This paper introduces the Causal Testing Framework: a framework that uses Causal Inference techniques to establish causal effects from existing data, enabling users to conduct software testing activities concerning the effect of a change, such as Metamorphic Testing, a posteriori. We present three case studies covering real-world scientific models, demonstrating how the Causal Testing Framework can infer metamorphic test outcomes from reused, confounded test data to provide an efficient solution for testing scientific modelling software.Keywords
All Related Versions
This publication has 49 references indexed in Scilit:
- Causal program dependence analysisScience of Computer Programming, 2024
- Analytical modelling and UK Government policyAI and Ethics, 2021
- Causability and explainability of artificial intelligence in medicineWIREs Data Mining and Knowledge Discovery, 2019
- Global Sensitivity Analysis of environmental models: Convergence and validationEnvironmental Modelling & Software, 2016
- Inferring extended finite state machine models from software executionsEmpirical Software Engineering, 2015
- Randomised controlled trials for Ebola: practical and ethical issuesThe Lancet, 2014
- Latin hypercube sampling with multidimensional uniformityJournal of Statistical Planning and Inference, 2012
- Bayesian emulation of complex multi-output and dynamic computer modelsJournal of Statistical Planning and Inference, 2010
- Using machine learning to refine Category-Partition test specifications and test suitesInformation and Software Technology, 2009
- Graphviz— Open Source Graph Drawing Tools2002