JWJoseph Wonsil

Joseph Wonsil

*This is an algorithmically generated author record based on publicly available information.

Current Organizations / Institutes

The organizations/institutes from the scholar's latest article


Joseph Wonsil has a total of 12 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • Open Access

    Raising the Reproducibility Bar

    by Joseph Wonsil, Rúbia Guerra, Adam Pocock, Jack Sullivan, Margo Seltzer


  • Open Access

    Computational Experiment Comprehension using Provenance Summarization

    by Nichole Boufford, Joseph Wonsil, Adam Pocock, Jack Sullivan, Margo Seltzer, Thomas Pasquier

    Scientists use complex multistep workflows to analyze data. However, reproducing computational experiments is often difficult as scientists’ software engineering practices are geared towards the science, not the programming. In particular, reproducing a scientific workflow frequently requires information about its execution. This information includes the precise versions of packages and libraries used, the particular processor used to perform floating point computation, and the language runtime used. This can be extracted from data provenance, the formal record of what happened during an experiment. However, data provenance is inherently graph-structured and often large, which makes interpretation challenging. Rather than exposing data provenance through its graphical representation, we propose a textual one and use a large language model to generate it. We develop techniques for prompting large language models to automatically generate textual summaries of provenance data. We conduct a user study to compare the effectiveness of these summaries to the more common node-link diagram representation. Study participants are able to extract useful information from both the textual summaries and node-link diagrams. The textual summaries were particularly beneficial for scientists with low computational expertise. We discuss the qualitative results from our study to motivate future designs for reproducibility tools.


  • Open Access

    Integrated Reproducibility with Self-describing Machine Learning Models

    by Joseph Wonsil, Jack Sullivan, Margo Seltzer, Adam Pocock

    Researchers and data scientists frequently want to collaborate on machine learning models. However, in the presence of sharing and simultaneous experimentation, it is challenging both to determine if two models were trained identically and to reproduce precisely someone else’s training process. We demonstrate how provenance collection that is tightly integrated into a machine learning library facilitates reproducibility. We present MERIT, a reproducibility system that leverages a robust configuration system and extensive provenance collection to exactly reproduce models, given only a model object. We integrate MERIT with Tribuo, an open-source Java-based machine learning library. Key features of this integrated reproducibility framework include controlling for sources of non-determinism in a multi-threaded environment and exposing the training differences between two models in a human-readable form. Our system allows simple reproduction of deployed Tribuo models without any additional information, ensuring data science research is reproducible. Our framework is open-source and available under an Apache 2.0 license.


  • Open Access

    Reproducibility as a service

    by Joseph Wonsil, Nichole Boufford, Prakhar Agrawal, Christopher Chen, Tianhang Cui, Akash Sivaram, Margo Seltzer

    Recent studies demonstrated that the reproducibility of previously published computational experiments is inadequate. Many of these published computational experiments never recorded or preserved their computational environment, including packages installed in the language, libraries installed on the host system, and file locations. Researchers have created reproducibility tools to help mitigate this problem, but these tools assume the experiment currently executes. Thus, these tools do not facilitate reproducibility of the large number of published experiments. This situation is not improving; researchers continue to publish without using reproducibility tools. We define a framework to distinguish between actions taken by a researcher to facilitate reproducibility in the presence of a computational environment and actions taken by a researcher to enable reproduction of an experiment when that environment has been lost to clarify the gap between what existing reproducibility tools are capable of and what is required to reproduce published experiments. The difference between these approaches lies in the availability of a computational environment. Researchers that provide access to the original computational environment perform proactive reproducibility, while those who do not enable only retroactive reproducibility. We present Reproducibility as a Service (RaaS), which is, to the best of our knowledge, the first reproducibility tool explicitly designed to facilitate retroactive reproducibility. We demonstrate how RaaS fixes many common errors found in R scripts on Harvard's Dataverse and preserves a recreated computational environment. Finally, we discuss how a retroactive reproducibility service such as RaaS is also helpful as an ‘artifact evaluation assistant’ in a journal's publication pipeline.