Joseph Wonsil
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Joseph Wonsil has a total of 12 co-authors. Here are the top co-authors by number of shared publications:
- Margo Seltzer
University of British Columbia, Vancouver, Canada
4 shared publications - Adam Pocock
Oracle Labs, Burlington, Massachusetts, USA
3 shared publications - Jack Sullivan
Oracle Labs, Burlington, Massachusetts, USA
3 shared publications - Nichole Boufford
Oracle Labs, Vancouver, British Columbia, Canada
2 shared publications Akash Sivaram
Department of Computer Science and Engineering Indian Institute of Technology Delhi Delhi India
1 shared publicationChristopher Chen
John A. Paulson School of Engineering and Applied Sciences Harvard University Massachusetts Cambridge USA
1 shared publication
Latest Publications
- Open Access
- 29 July 2025
- Published by Association for Computing Machinery (ACM)
- p. 234-242
Raising the Reproducibility Bar
- 29 July 2025
- Published by Association for Computing Machinery (ACM)
- p. 234-242
Open AccessRaising the Reproducibility Bar
- Open Access
- 29 July 2025
- Published by Association for Computing Machinery (ACM)
- p. 248-249
Experience with Reproducibility and Consistency in Writing an Academic Paper
- 29 July 2025
- Published by Association for Computing Machinery (ACM)
- p. 248-249
Open AccessExperience with Reproducibility and Consistency in Writing an Academic Paper
- Open Access
- 18 June 2024
- Published by Association for Computing Machinery (ACM)
- p. 1-19
Computational Experiment Comprehension using Provenance Summarization
Scientists use complex multistep workflows to analyze data. However, reproducing computational experiments is often difficult as scientists’ software engineering practices are geared towards the science, not the programming. In particular, reproducing a scientific workflow frequently requires information about its execution. This information includes the precise versions of packages and libraries used, the particular processor used to perform floating point computation, and the language runtime used. This can be extracted from data provenance, the formal record of what happened during an experiment. However, data provenance is inherently graph-structured and often large, which makes interpretation challenging. Rather than exposing data provenance through its graphical representation, we propose a textual one and use a large language model to generate it. We develop techniques for prompting large language models to automatically generate textual summaries of provenance data. We conduct a user study to compare the effectiveness of these summaries to the more common node-link diagram representation. Study participants are able to extract useful information from both the textual summaries and node-link diagrams. The textual summaries were particularly beneficial for scientists with low computational expertise. We discuss the qualitative results from our study to motivate future designs for reproducibility tools.
- 18 June 2024
- Published by Association for Computing Machinery (ACM)
- p. 1-19
Open AccessComputational Experiment Comprehension using Provenance Summarization
Scientists use complex multistep workflows to analyze data. However, reproducing computational experiments is often difficult as scientists’ software engineering practices are geared towards the science, not the programming. In particular, reproducing a scientific workflow frequently requires information about its execution. This information includes the precise versions of packages and libraries used, the particular processor used to perform floating point computation, and the language runtime used. This can be extracted from data provenance, the formal record of what happened during an experiment. However, data provenance is inherently graph-structured and often large, which makes interpretation challenging. Rather than exposing data provenance through its graphical representation, we propose a textual one and use a large language model to generate it. We develop techniques for prompting large language models to automatically generate textual summaries of provenance data. We conduct a user study to compare the effectiveness of these summaries to the more common node-link diagram representation. Study participants are able to extract useful information from both the textual summaries and node-link diagrams. The textual summaries were particularly beneficial for scientists with low computational expertise. We discuss the qualitative results from our study to motivate future designs for reproducibility tools.
- Open Access
- 27 June 2023
- Published by Association for Computing Machinery (ACM)
- p. 1-14
Integrated Reproducibility with Self-describing Machine Learning Models
Researchers and data scientists frequently want to collaborate on machine learning models. However, in the presence of sharing and simultaneous experimentation, it is challenging both to determine if two models were trained identically and to reproduce precisely someone else’s training process. We demonstrate how provenance collection that is tightly integrated into a machine learning library facilitates reproducibility. We present MERIT, a reproducibility system that leverages a robust configuration system and extensive provenance collection to exactly reproduce models, given only a model object. We integrate MERIT with Tribuo, an open-source Java-based machine learning library. Key features of this integrated reproducibility framework include controlling for sources of non-determinism in a multi-threaded environment and exposing the training differences between two models in a human-readable form. Our system allows simple reproduction of deployed Tribuo models without any additional information, ensuring data science research is reproducible. Our framework is open-source and available under an Apache 2.0 license.
- 27 June 2023
- Published by Association for Computing Machinery (ACM)
- p. 1-14
Open AccessIntegrated Reproducibility with Self-describing Machine Learning Models
Researchers and data scientists frequently want to collaborate on machine learning models. However, in the presence of sharing and simultaneous experimentation, it is challenging both to determine if two models were trained identically and to reproduce precisely someone else’s training process. We demonstrate how provenance collection that is tightly integrated into a machine learning library facilitates reproducibility. We present MERIT, a reproducibility system that leverages a robust configuration system and extensive provenance collection to exactly reproduce models, given only a model object. We integrate MERIT with Tribuo, an open-source Java-based machine learning library. Key features of this integrated reproducibility framework include controlling for sources of non-determinism in a multi-threaded environment and exposing the training differences between two models in a human-readable form. Our system allows simple reproduction of deployed Tribuo models without any additional information, ensuring data science research is reproducible. Our framework is open-source and available under an Apache 2.0 license.
- Open Access
- 21 March 2023
- Published by Wiley in Software: Practice and Experience
- Vol. 53 (7) , 1543-1571
Reproducibility as a service
Recent studies demonstrated that the reproducibility of previously published computational experiments is inadequate. Many of these published computational experiments never recorded or preserved their computational environment, including packages installed in the language, libraries installed on the host system, and file locations. Researchers have created reproducibility tools to help mitigate this problem, but these tools assume the experiment currently executes. Thus, these tools do not facilitate reproducibility of the large number of published experiments. This situation is not improving; researchers continue to publish without using reproducibility tools. We define a framework to distinguish between actions taken by a researcher to facilitate reproducibility in the presence of a computational environment and actions taken by a researcher to enable reproduction of an experiment when that environment has been lost to clarify the gap between what existing reproducibility tools are capable of and what is required to reproduce published experiments. The difference between these approaches lies in the availability of a computational environment. Researchers that provide access to the original computational environment perform proactive reproducibility, while those who do not enable only retroactive reproducibility. We present Reproducibility as a Service (RaaS), which is, to the best of our knowledge, the first reproducibility tool explicitly designed to facilitate retroactive reproducibility. We demonstrate how RaaS fixes many common errors found in R scripts on Harvard's Dataverse and preserves a recreated computational environment. Finally, we discuss how a retroactive reproducibility service such as RaaS is also helpful as an ‘artifact evaluation assistant’ in a journal's publication pipeline.
- 21 March 2023
- Published by Wiley in Software: Practice and Experience
- Vol. 53 (7) , 1543-1571
Open AccessReproducibility as a service
Recent studies demonstrated that the reproducibility of previously published computational experiments is inadequate. Many of these published computational experiments never recorded or preserved their computational environment, including packages installed in the language, libraries installed on the host system, and file locations. Researchers have created reproducibility tools to help mitigate this problem, but these tools assume the experiment currently executes. Thus, these tools do not facilitate reproducibility of the large number of published experiments. This situation is not improving; researchers continue to publish without using reproducibility tools. We define a framework to distinguish between actions taken by a researcher to facilitate reproducibility in the presence of a computational environment and actions taken by a researcher to enable reproduction of an experiment when that environment has been lost to clarify the gap between what existing reproducibility tools are capable of and what is required to reproduce published experiments. The difference between these approaches lies in the availability of a computational environment. Researchers that provide access to the original computational environment perform proactive reproducibility, while those who do not enable only retroactive reproducibility. We present Reproducibility as a Service (RaaS), which is, to the best of our knowledge, the first reproducibility tool explicitly designed to facilitate retroactive reproducibility. We demonstrate how RaaS fixes many common errors found in R scripts on Harvard's Dataverse and preserves a recreated computational environment. Finally, we discuss how a retroactive reproducibility service such as RaaS is also helpful as an ‘artifact evaluation assistant’ in a journal's publication pipeline.