Reed Milewicz

Open Source Software (OSS) communities face a wave of uncertainty as Generative AI (GenAI) rapidly transforms how software is created, maintained, and governed. Without clear frameworks, communities risk being overwhelmed by the complexity and ambiguity introduced by GenAI, threatening the collaborative ethos that underpins OSS. To address this gap, we present a Socio-Technical System (STS)-guided conceptual framework that applies McLuhan's Tetrad as an analytic lens to articulate how GenAI reshapes the socio-technical dynamics of OSS development. Through a scenario-based exploration across four components of the STS-guided framework—software practices, documentation, community engagement, and governance—we identify plausible socio-technical impacts and outline a corresponding Roadmap for sustaining OSS communities in the Age of GenAI. This Roadmap will enable OSS researchers and practitioners to interpret emerging transformations, anticipate challenges, and design interventions that foster long-term community resilience. By adopting this framework, OSS leaders and researchers can proactively shape the future of their ecosystems, rather than simply reacting to technological upheaval.

Scientific software is essential to scientific innovation and in many ways it is distinct from other types of software. Abandoned (or unmaintained), buggy, and hard to use software, a perception often associated with scientific software can hinder scientific progress, yet, in contrast to other types of software, its longevity is poorly understood. Existing data curation efforts are fragmented by science domain and/or are small in scale and lack key attributes. We use large language models to classify public software repositories in World of Code into distinct scientific domains and layers of the software stack, curating a large and diverse collection of over 18,000 scientific software projects. Using this data, we estimate survival models to understand how the domain, infrastructural layer, and other attributes of scientific software affect its longevity. We further obtain a matched sample of non-scientific software repositories and investigate the differences. We find that infrastructural layers, downstream dependencies, mentions of publications, and participants from government are associated with a longer lifespan, while newer projects with participants from academia had shorter lifespan. Against common expectations, scientific projects have a longer lifetime than matched non-scientific open-source software projects. We expect our curated attribute-rich collection to support future research on scientific software and provide insights that may help extend longevity of both scientific and other projects.

Mentorship in open source software (OSS) is a vital, multifaceted process that includes onboarding newcomers, fostering skill development, and enhancing community building. This study examines task-focused mentoring strategies that help mentees complete their tasks and the ideal personal qualities and outcomes of good mentorship in OSS communities. We conducted two surveys to gather contributor perceptions: the first survey, with 70 mentors, mapped 17 mentoring challenges to 21 strategies that help support mentees. The second survey, with 85 contributors, assessed the importance of personal qualities and ideal mentorship outcomes. Our findings not only provide actionable strategies to help mentees overcome challenges and become successful contributors but also guide current and future mentors and OSS communities in understanding the personal qualities that are the cornerstone of good mentorship and the outcomes that mentor-mentee pairs should aspire to achieve.

Scientific modeling and simulation software is ubiquitous at Sandia National Laboratories and is integral to providing empirical justification to critical mission decisions. Models are increasingly being expressed as workflows to simplify the many steps needed in scientific analyses but keeping these models and workflows alive for the decades-long timescales needed by Sandia remains a struggle. Additionally, the manual use and (lack of) maintenance of these models creates significant risks for duplicated work and model capability loss over time from changing personnel and computing environments. To address these issues, we are building the Engineering Common Modeling Framework (ECMF), a platform for scientific model sustainment at Sandia. ECMF enables the automatic evaluation of models over time and will ensure that models created at Sandia are discoverable and ready to be revisited, extended, and reused. In this paper, we report our current and planned capabilities as well as lessons learned from our framework development process.

Among computational science and engineering (CSE) software development projects, reproducibility of results is widely understood to be essential for the software to be useful and meaningful. At present, there is a lack of empirical support for how best to design for, implement, or maintain reproducibility over the course of the CSE software lifecycle, and the relationship between reproducibility and other qualities of interest is not well-understood. In this study, we consider the role of software engineering practices and product qualities in enabling reproducibility. To build a foundation for future studies, we conducted case studies of the software engineering practices of 9 teams at Sandia National Laboratories (Sandia), and a first-in-kind online survey of 219 developers and users of CSE software to assess their beliefs about quality and reproducibility. Among our results, we identified three practices that were universally attributed to enhancing reproducibility of research and software at Sandia: automated testing, documentation, and version control. We found that the majority of survey respondents believed that high-quality software was more likely to be reproducible than low-quality software, and that maintainability, reliability, and usability were generally seen as positively contributing to reproducibility. These preliminary findings merit follow-up studies to better understand how software quality is valued and enacted in CSE software development practice.

Computational provenance has many important applications, especially to reproducibility. System-level provenance collectors can track provenance data without requiring the user to change anything about their application. However, system-level provenance collectors have performance overheads, and, worse still, different works use different and incomparable benchmarks to assess their performance overhead. This work identifies user-space system-level provenance collectors in prior work, collates the benchmarks, and evaluates each collector on each benchmark. We use benchmark minimization to select a minimal subset of benchmarks, which can be used as goalposts for future work on system-level provenance collectors.