Graham Williams
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Research Subjects
Research Interests
Graham Williams has a total of 94 co-authors. Here are the top co-authors by number of shared publications:
- Alex Potanin
Australian National University
5 shared publications Hongxing He
CSIRO Mathematical and Information Sciences
5 shared publications- Peter Christen
School of Computing, Australian National University
5 shared publications - Zara Hassan
Australian National University, Canberra, Australia
5 shared publications - Christoph Treude
Singapore Management University, Singapore
4 shared publications - Michael Norrish
Australian National University, Canberra, Australia
4 shared publications
Latest Publications
- Open Access
- 3 November 2026
- Published by Springer Nature in Empirical Software Engineering
- Vol. 32 (2) , 54
Reproducibility debt: insights from practitioners
Context: Reproducibility Debt (RpD) refers to the accumulative issues and challenges that hinder the ability to reproduce scientific outcomes, stemming from challenges inherent in scientific software code and data, including development, organisation, dissemination, and documentation. RpD may accumulate when researchers and scientific software practitioners engage in sub-optimal activities, often for short-term benefits, which ultimately compromise the reproducibility of research outcomes. Hence, it is essential to understand the factors that could lead scientific software teams to incur RpD and the strategies they adopt to prevent RpD in their projects. Objective: This research aims to understand practitioners’ perspectives on RpD, the factors they perceive as contributing to it, their consequences, and the practices they adopt to address it. Methodology: We present a mixed-methods study combining exploratory interviews with scientific software practitioners and a complementary survey conducted using our InsightRpD survey tool. Results: In total, 23 practitioners participated in interviews and 59 completed the survey. We identified 69 practitioner-reported causes and 80 practitioner-reported effects associated with RpD, confirming and extending current knowledge. Of the 38 causes reported in our previous Systematic Literature Review (SLR), 31 were supported by evidence from this study. The empirical data also highlights 38 additional causes and 47 additional effects not captured in prior literature, which are associated with evolving software practices, such as automation, decentralised teams, and the use of generative AI. Additionally, we explore the mitigation strategies employed by practitioners to address and reduce the impact of RpD. Conclusion: By identifying factors contributing to RpD, its associated consequences, and mitigation strategies from practitioner insights, this study provides evidence-based diagrams of practitioner-reported cause-effect relationships and practical approaches that advance the understanding and management of RpD, while informing future research and practice in Research Software Engineering (RSE) and software sustainability.
- 3 November 2026
- Published by Springer Nature in Empirical Software Engineering
- Vol. 32 (2) , 54
Open AccessReproducibility debt: insights from practitioners
Context: Reproducibility Debt (RpD) refers to the accumulative issues and challenges that hinder the ability to reproduce scientific outcomes, stemming from challenges inherent in scientific software code and data, including development, organisation, dissemination, and documentation. RpD may accumulate when researchers and scientific software practitioners engage in sub-optimal activities, often for short-term benefits, which ultimately compromise the reproducibility of research outcomes. Hence, it is essential to understand the factors that could lead scientific software teams to incur RpD and the strategies they adopt to prevent RpD in their projects. Objective: This research aims to understand practitioners’ perspectives on RpD, the factors they perceive as contributing to it, their consequences, and the practices they adopt to address it. Methodology: We present a mixed-methods study combining exploratory interviews with scientific software practitioners and a complementary survey conducted using our InsightRpD survey tool. Results: In total, 23 practitioners participated in interviews and 59 completed the survey. We identified 69 practitioner-reported causes and 80 practitioner-reported effects associated with RpD, confirming and extending current knowledge. Of the 38 causes reported in our previous Systematic Literature Review (SLR), 31 were supported by evidence from this study. The empirical data also highlights 38 additional causes and 47 additional effects not captured in prior literature, which are associated with evolving software practices, such as automation, decentralised teams, and the use of generative AI. Additionally, we explore the mitigation strategies employed by practitioners to address and reduce the impact of RpD. Conclusion: By identifying factors contributing to RpD, its associated consequences, and mitigation strategies from practitioner insights, this study provides evidence-based diagrams of practitioner-reported cause-effect relationships and practical approaches that advance the understanding and management of RpD, while informing future research and practice in Research Software Engineering (RSE) and software sustainability.
- Open Access
- 12 April 2026
- Published by Association for Computing Machinery (ACM)
- p. 23-24
Managing Reproducibility Debt in Scientific Software: A Practical Framework
- 12 April 2026
- Published by Association for Computing Machinery (ACM)
- p. 23-24
Open AccessManaging Reproducibility Debt in Scientific Software: A Practical Framework
- 13 October 2025
- Published by Springer Nature
- p. 151-155
Taxonomy of Anomaly Types in Knowledge Graphs
- Open Access
- 12 October 2025
- Published by Association for Computing Machinery (ACM)
- p. 50-51
Reproducibility Debt in Scientific Software
- 12 October 2025
- Published by Association for Computing Machinery (ACM)
- p. 50-51
Open AccessReproducibility Debt in Scientific Software
- Open Access
- 7 August 2025
- Published by IOS Press
- Vol. 329, 2012-2013
Co-Designing a Multiple Sclerosis Fatigue App
Despite fatigue being a common and debilitating symptom of Multiple Sclerosis (MS), it is poorly understood. Here we report on facilitated workshops and surveys to capture the needs and preferences of people with MS to inform further work towards fatigue survey Apps.
- 7 August 2025
- Published by IOS Press
- Vol. 329, 2012-2013
Open AccessCo-Designing a Multiple Sclerosis Fatigue App
Despite fatigue being a common and debilitating symptom of Multiple Sclerosis (MS), it is poorly understood. Here we report on facilitated workshops and surveys to capture the needs and preferences of people with MS to inform further work towards fatigue survey Apps.
- Open Access
- 26 December 2024
- Published by Elsevier in Journal of Systems and Software
- Vol. 222
Characterising reproducibility debt in scientific software: A systematic literature review
- 26 December 2024
- Published by Elsevier in Journal of Systems and Software
- Vol. 222
Open AccessCharacterising reproducibility debt in scientific software: A systematic literature review
- Open Access
Anomaly Detection and Classification in Knowledge Graphs
Anomalies such as redundant, inconsistent, contradictory, and deficient values in a Knowledge Graph (KG) are unavoidable, as these graphs are often curated manually, or extracted using machine learning and natural language processing techniques. Therefore, anomaly detection is a task that can enhance the quality of KGs. In this paper, we propose SEKA (SEeking Knowledge graph Anomalies), an unsupervised approach for the detection of abnormal triples and entities in KGs. SEKA can help improve the correctness of a KG whilst retaining its coverage. We propose an adaption of the Path Rank Algorithm (PRA), named the Corroborative Path Rank Algorithm (CPRA), which is an efficient adaptation of PRA that is customized to detect anomalies in KGs. Furthermore, we also present TAXO (TAXOnomy of anomaly types in KGs), a taxonomy of possible anomaly types that can occur in a KG. This taxonomy provides a classification of the anomalies discovered by SEKA with an extensive discussion of possible data quality issues in a KG. We evaluate both approaches using the four real-world KGs YAGO-1, KBpedia, Wikidata, and DSKG to demonstrate the ability of SEKA and TAXO to outperform the baselines.
Open AccessAnomaly Detection and Classification in Knowledge Graphs
Anomalies such as redundant, inconsistent, contradictory, and deficient values in a Knowledge Graph (KG) are unavoidable, as these graphs are often curated manually, or extracted using machine learning and natural language processing techniques. Therefore, anomaly detection is a task that can enhance the quality of KGs. In this paper, we propose SEKA (SEeking Knowledge graph Anomalies), an unsupervised approach for the detection of abnormal triples and entities in KGs. SEKA can help improve the correctness of a KG whilst retaining its coverage. We propose an adaption of the Path Rank Algorithm (PRA), named the Corroborative Path Rank Algorithm (CPRA), which is an efficient adaptation of PRA that is customized to detect anomalies in KGs. Furthermore, we also present TAXO (TAXOnomy of anomaly types in KGs), a taxonomy of possible anomaly types that can occur in a KG. This taxonomy provides a classification of the anomalies discovered by SEKA with an extensive discussion of possible data quality issues in a KG. We evaluate both approaches using the four real-world KGs YAGO-1, KBpedia, Wikidata, and DSKG to demonstrate the ability of SEKA and TAXO to outperform the baselines.
- Open Access
- 10 July 2024
- Published by Association for Computing Machinery (ACM)
- p. 462-466
Reproducibility Debt: Challenges and Future Pathways
Reproducibility of scientific computation is a critical factor in validating its underlying process, but it is often elusive. Complexity and continuous evolution in software systems have introduced new challenges for reproducibility across a myriad of computational sciences, resulting in growing debt. This requires a comprehensive domain-agnostic study to define and asses Reproducibility Debt (RpD) in scientific software, thus uncovering and classifying all underlying factors attributed towards its emergence and identification i.e., causes and effects. Moreover, an organised map of prevention strategies is imperative to guide researchers for its proactive management. This vision paper highlights the challenges that hinder effective management of RpD in scientific software, with preliminary results from our ongoing work and an agenda for future research.
- 10 July 2024
- Published by Association for Computing Machinery (ACM)
- p. 462-466
Open AccessReproducibility Debt: Challenges and Future Pathways
Reproducibility of scientific computation is a critical factor in validating its underlying process, but it is often elusive. Complexity and continuous evolution in software systems have introduced new challenges for reproducibility across a myriad of computational sciences, resulting in growing debt. This requires a comprehensive domain-agnostic study to define and asses Reproducibility Debt (RpD) in scientific software, thus uncovering and classifying all underlying factors attributed towards its emergence and identification i.e., causes and effects. Moreover, an organised map of prevention strategies is imperative to guide researchers for its proactive management. This vision paper highlights the challenges that hinder effective management of RpD in scientific software, with preliminary results from our ongoing work and an agenda for future research.
- Open Access
- 28 December 2022
- Published by Association for Computing Machinery (ACM) in Journal of Data and Information Quality
- Vol. 15 (1) , 1-37
Unsupervised Identification of Abnormal Nodes and Edges in Graphs
Much of today’s data are represented as graphs, ranging from social networks to bibliographic citations. Nodes in such graphs correspond to records that generally represent entities, while edges represent relationships between these entities. Both nodes and edges in a graph can have attributes that characterize the entities and their relationships. Relationships are either explicitly known (like friends in a social network), or they are inferred using link prediction (such as two babies are siblings because they have the same mother). Any graph representing real-world data likely contains nodes and edges that are abnormal, and identifying these can be important for outlier detection in applications ranging from crime and fraud detection to viral marketing. We propose a novel approach to the unsupervised detection of abnormal nodes and edges in graphs. We first characterize nodes and edges using a set of features, and then employ a one-class classifier to identify abnormal nodes and edges. We extract patterns of features from these abnormal nodes and edges, and apply clustering to identify groups of patterns with similar characteristics. We finally visualize these abnormal patterns to show co-occurrences of features and relationships between those features that mostly influence the abnormality of nodes and edges. We evaluate our approach on datasets from diverse domains, including historical birth certificates, COVID patient records, e-mails, books, and movies. This evaluation demonstrates that our approach is well suited to identify both abnormal nodes and edges in graphs in an unsupervised way, and it can outperform several baseline anomaly detection techniques.
- 28 December 2022
- Published by Association for Computing Machinery (ACM) in Journal of Data and Information Quality
- Vol. 15 (1) , 1-37
Open AccessUnsupervised Identification of Abnormal Nodes and Edges in Graphs
Much of today’s data are represented as graphs, ranging from social networks to bibliographic citations. Nodes in such graphs correspond to records that generally represent entities, while edges represent relationships between these entities. Both nodes and edges in a graph can have attributes that characterize the entities and their relationships. Relationships are either explicitly known (like friends in a social network), or they are inferred using link prediction (such as two babies are siblings because they have the same mother). Any graph representing real-world data likely contains nodes and edges that are abnormal, and identifying these can be important for outlier detection in applications ranging from crime and fraud detection to viral marketing. We propose a novel approach to the unsupervised detection of abnormal nodes and edges in graphs. We first characterize nodes and edges using a set of features, and then employ a one-class classifier to identify abnormal nodes and edges. We extract patterns of features from these abnormal nodes and edges, and apply clustering to identify groups of patterns with similar characteristics. We finally visualize these abnormal patterns to show co-occurrences of features and relationships between those features that mostly influence the abnormality of nodes and edges. We evaluate our approach on datasets from diverse domains, including historical birth certificates, COVID patient records, e-mails, books, and movies. This evaluation demonstrates that our approach is well suited to identify both abnormal nodes and edges in graphs in an unsupervised way, and it can outperform several baseline anomaly detection techniques.