Jeremy Fischer

Over the past decade, the convergence of Cloud and High-Performance Computing (HPC) has undergone significant movement. We explore the evolution, motivations, and practicalities of establishing on-premise research cloud infrastructure and the complementary nature with HPC and commercial resources; under the belief that research clouds serve a unique role within research and education as a convergence accelerator. This role is highlighted through exploring the design tradeoffs in architecting research clouds versus HPC resources, focusing on the balance between utility, availability, and hardware utilization. The discussion provides insights from experiences with the National Science Foundation-supported Jetstream and Jetstream2 systems, showcasing convergence technologies and challenges. A variety of real-world use cases are provided that show the interplay between these computing paradigms; exploring use in research and education for interactive and iterative development, as an on-ramp to large-scale resources, as a powerful tool for education and workforce development, and for domain specific science gateways.

Neuroscience is advancing standardization and tool development to support rigor and transparency. Consequently, data pipeline complexity has increased, hindering FAIR (findable, accessible, interoperable and reusable) access. brainlife.io was developed to democratize neuroimaging research. The platform provides data standardization, management, visualization and processing and automatically tracks the provenance history of thousands of data objects. Here, brainlife.io is described and evaluated for validity, reliability, reproducibility, replicability and scientific utility using four data modalities and 3,200 participants.

This paper uses accounting concepts—particularly the concept of Return on Investment (ROI)—to reveal the quantitative value of scientific research pertaining to a major US cyberinfrastructure project (XSEDE—the eXtreme Science and Engineering Discovery Environment). XSEDE provides operational and support services for advanced information technology systems, cloud systems, and supercomputers supporting non-classified US research, with an average budget for XSEDE of US$20M+ per year over the period studied (2014–2021). To assess the financial effectiveness of these services, we calculated a proxy for ROI, and converted quantitative measures of XSEDE service delivery into financial values using costs for service from the US marketplace. We calculated two estimates of ROI: a Conservative Estimate, functioning as a lower bound and using publicly available data for a lower valuation of XSEDE services; and a Best Available Estimate, functioning as a more accurate estimate, but using some unpublished valuation data. Using the largest dataset assembled for analysis of ROI for a cyberinfrastructure project, we found a Conservative Estimate of ROI of 1.87, and a Best Available Estimate of ROI of 3.24. Through accounting methods, we show that XSEDE services offer excellent value to the US government, that the services offered uniquely by XSEDE (that is, not otherwise available for purchase) were the most valuable to the facilitation of US research activities, and that accounting-based concepts hold great value for understanding the mechanisms of scientific research generally.

Jetstream ¹ [45] is a first-of-a-kind cloud system funded by the National Science Foundation and by contributions from the several participants that created and now operate this system. It is a first-of-a-kind system in the sense that it is the first cloud system funded by the NSF accessible to and designed to be a resource for the national research community. Jetstream is also in a real sense a pilot project because the NSF has never before funded the creation of a cloud system. Proposing and implementing Jetstream was at times very much a process of putting a square peg into a round hole. Jetstream was funded through an NSF high performance computing (HPC) system acquisition solicitation, but the actual acquisition of the hardware from the vendor (Dell) was a small minority of the effort needed to put a cloud system into production.

Summary: Jetstream is a first of its kind system for the NSF — a distributed production cloud resource. We review the purpose for creating Jetstream, discuss Jetstream's key characteristics, describe our experiences from the first year of maintaining an OpenStack‐based cloud environment, and share some of the early scientific impacts achieved by Jetstream users. Jetstream offers a unique capability within the XSEDE‐supported US national cyberinfrastructure, delivering interactive virtual machines (VMs) via the Atmosphere interface. As a multi‐region deployment that operates as an integrated system, Jetstream is proving effective in supporting modes and disciplines of research traditionally underrepresented on larger XSEDE‐supported clusters and supercomputers. Already, Jetstream has been used to perform research and education in biology, biochemistry, atmospheric science, earth science, and computer science.