Robert Latham
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Robert Latham has a total of 89 co-authors. Here are the top co-authors by number of shared publications:
- Robert Ross
Argonne National Laboratory, Lemont, USA
7 shared publications - Philip Carns
Argonne National Laboratory (ANL), Lemont, USA
6 shared publications Rob Ross
Argonne National Laboratory Argonne IL 60439 USA
5 shared publicationsRobert Ross
Argonne National Laboratory
5 shared publications- Shane Snyder
Argonne National Laboratory, Lemont, IL, USA
5 shared publications - Matthieu Dorier
Argonne National Laboratory, Lemont, IL, USA
4 shared publications
Latest Publications
- Open Access
- 15 November 2025
- Published by Association for Computing Machinery (ACM)
- p. 1385-1395
Parallel Data Object Creation: Scalable Metadata Management in Parallel I/O Library
- 15 November 2025
- Published by Association for Computing Machinery (ACM)
- p. 1385-1395
Open AccessParallel Data Object Creation: Scalable Metadata Management in Parallel I/O Library
- Open Access
- 15 November 2025
- Published by Association for Computing Machinery (ACM)
- p. 2281-2292
RESILIO : A Scalable and Composable Architecture for Tomographic Reconstruction Workflows
- 15 November 2025
- Published by Association for Computing Machinery (ACM)
- p. 2281-2292
Open AccessRESILIO : A Scalable and Composable Architecture for Tomographic Reconstruction Workflows
- Open Access
Parallel Data Object Creation: Towards Scalable Metadata Management in High-Performance I/O Library
High-level I/O libraries, such as HDF5 and PnetCDF, are commonly used by large-scale scientific applications to perform I/O tasks in parallel. These I/O libraries store the metadata such as data types and dimensionality along with the raw data in the same files. While these libraries are well-optimized for concurrent access to the raw data, they are designed neither to handle a large number of data objects efficiently nor to create different data objects independently by multiple processes, as they require applications to call data object creation APIs collectively with consistent metadata among all processes. Applications that process data gathered from remote sensors, such as particle collision experiments in high-energy physics, may generate data of different sizes from different sensors and desire to store them as separate data objects. For such applications, the I/O library's requirement on collective data object creation can become very expensive, as the cost of metadata consistency check increases with the metadata volume as well as the number of processes. To address this limitation, using PnetCDF as an experimental platform, we investigate solutions in this paper that abide the netCDF file format, as well as propose a new file header format that enables independent data object creation. The proposed file header consists of two sections, an index table and a list of metadata blocks. The index table contains the reference to the metadata blocks and each block stores metadata of objects that can be created collectively or independently. The new design achieves a scalable performance, cutting data object creation times by up to 582x when running on 4096 MPI processes to create 5,684,800 data objects in parallel. Additionally, the new method reduces the memory footprints, with each process requiring an amount of memory space inversely proportional to the number of processes.
Open AccessParallel Data Object Creation: Towards Scalable Metadata Management in High-Performance I/O Library
High-level I/O libraries, such as HDF5 and PnetCDF, are commonly used by large-scale scientific applications to perform I/O tasks in parallel. These I/O libraries store the metadata such as data types and dimensionality along with the raw data in the same files. While these libraries are well-optimized for concurrent access to the raw data, they are designed neither to handle a large number of data objects efficiently nor to create different data objects independently by multiple processes, as they require applications to call data object creation APIs collectively with consistent metadata among all processes. Applications that process data gathered from remote sensors, such as particle collision experiments in high-energy physics, may generate data of different sizes from different sensors and desire to store them as separate data objects. For such applications, the I/O library's requirement on collective data object creation can become very expensive, as the cost of metadata consistency check increases with the metadata volume as well as the number of processes. To address this limitation, using PnetCDF as an experimental platform, we investigate solutions in this paper that abide the netCDF file format, as well as propose a new file header format that enables independent data object creation. The proposed file header consists of two sections, an index table and a list of metadata blocks. The index table contains the reference to the metadata blocks and each block stores metadata of objects that can be created collectively or independently. The new design achieves a scalable performance, cutting data object creation times by up to 582x when running on 4096 MPI processes to create 5,684,800 data objects in parallel. Additionally, the new method reduces the memory footprints, with each process requiring an amount of memory space inversely proportional to the number of processes.
- Open Access
- 4 May 2025
- Published by Association for Computing Machinery (ACM)
- p. 37-49
Expanding Community Access to Real-World HPC Application I/O Characterization Data Using Darshan
- 4 May 2025
- Published by Association for Computing Machinery (ACM)
- p. 37-49
Open AccessExpanding Community Access to Real-World HPC Application I/O Characterization Data Using Darshan
- Open Access
- 4 May 2025
- Published by Association for Computing Machinery (ACM)
- p. 79-92
Towards Empirical Roofline Modeling of Distributed Data Services: Mapping the Boundaries of RPC Throughput
- 4 May 2025
- Published by Association for Computing Machinery (ACM)
- p. 79-92
Open AccessTowards Empirical Roofline Modeling of Distributed Data Services: Mapping the Boundaries of RPC Throughput
- 19 August 2024
- Published by SAGE Publications in The International Journal of High Performance Computing Applications
- Vol. 38 (5) , 527-545
Designing and prototyping extensions to the Message Passing Interface in MPICH
As HPC system architectures and the applications running on them continue to evolve, the MPI standard itself must evolve. The trend in current and future HPC systems toward powerful nodes with multiple CPU cores and multiple GPU accelerators makes efficient support for hybrid programming critical for applications to achieve high performance. However, the support for hybrid programming in the MPI standard has not kept up with recent trends. The MPICH implementation of MPI provides a platform for implementing and experimenting with new proposals and extensions to fill this gap and to gain valuable experience and feedback before the MPI Forum can consider them for standardization. In this work, we detail six extensions implemented in MPICH to increase MPI interoperability with other runtimes, with a specific focus on heterogeneous architectures. First, the extension to MPI generalized requests lets applications integrate asynchronous tasks into MPI’s progress engine. Second, the iovec extension to datatypes lets applications use MPI datatypes as a general-purpose data layout API beyond just MPI communications. Third, a new MPI object, MPIX_Stream, can be used by applications to identify execution contexts beyond MPI processes, including threads and GPU streams. MPIX stream communicators can be created to make existing MPI functions thread-aware and GPU-aware, thus providing applications with explicit ways to achieve higher performance. Fourth, MPIX Streams are extended to support the enqueue semantics for offloading MPI communications onto a GPU stream context. Fifth, thread communicators allow MPI communicators to be constructed with individual threads, thus providing a new level of interoperability between MPI and on-node runtimes such as OpenMP. Lastly, we present an extension to invoke MPI progress, which lets users spawn progress threads with fine-grained control to adapt the communication performance to their application designs. We describe the design and implementation of these extensions, provide usage examples, and highlight their expected benefits with performance results.
- 19 August 2024
- Published by SAGE Publications in The International Journal of High Performance Computing Applications
- Vol. 38 (5) , 527-545
Designing and prototyping extensions to the Message Passing Interface in MPICH
As HPC system architectures and the applications running on them continue to evolve, the MPI standard itself must evolve. The trend in current and future HPC systems toward powerful nodes with multiple CPU cores and multiple GPU accelerators makes efficient support for hybrid programming critical for applications to achieve high performance. However, the support for hybrid programming in the MPI standard has not kept up with recent trends. The MPICH implementation of MPI provides a platform for implementing and experimenting with new proposals and extensions to fill this gap and to gain valuable experience and feedback before the MPI Forum can consider them for standardization. In this work, we detail six extensions implemented in MPICH to increase MPI interoperability with other runtimes, with a specific focus on heterogeneous architectures. First, the extension to MPI generalized requests lets applications integrate asynchronous tasks into MPI’s progress engine. Second, the iovec extension to datatypes lets applications use MPI datatypes as a general-purpose data layout API beyond just MPI communications. Third, a new MPI object, MPIX_Stream, can be used by applications to identify execution contexts beyond MPI processes, including threads and GPU streams. MPIX stream communicators can be created to make existing MPI functions thread-aware and GPU-aware, thus providing applications with explicit ways to achieve higher performance. Fourth, MPIX Streams are extended to support the enqueue semantics for offloading MPI communications onto a GPU stream context. Fifth, thread communicators allow MPI communicators to be constructed with individual threads, thus providing a new level of interoperability between MPI and on-node runtimes such as OpenMP. Lastly, we present an extension to invoke MPI progress, which lets users spawn progress threads with fine-grained control to adapt the communication performance to their application designs. We describe the design and implementation of these extensions, provide usage examples, and highlight their expected benefits with performance results.
- Open Access
- 17 January 2024
- Published by American Physical Society (APS) in Physical Review D
- Vol. 109 (1) , 014013
Efficient precision simulation of processes with many-jet final states at the LHC
We present a scalable technique for the simulation of collider events with multijet final states, based on an improved parton-level event file format and scalable . The method is implemented for both leading- and next-to-leading-order QCD calculations. We perform a comprehensive analysis of the computing performance and validate our new framework using Higgs-boson plus multijet production with up to seven jets. We make the resulting code base available for public use.- 17 January 2024
- Published by American Physical Society (APS) in Physical Review D
- Vol. 109 (1) , 014013
Open AccessEfficient precision simulation of processes with many-jet final states at the LHC
We present a scalable technique for the simulation of collider events with multijet final states, based on an improved parton-level event file format and scalable . The method is implemented for both leading- and next-to-leading-order QCD calculations. We perform a comprehensive analysis of the computing performance and validate our new framework using Higgs-boson plus multijet production with up to seven jets. We make the resulting code base available for public use. - Open Access
- 12 November 2023
- Published by Association for Computing Machinery (ACM)
- p. 1380-1391
Enabling Agile Analysis of I/O Performance Data with PyDarshan
- 12 November 2023
- Published by Association for Computing Machinery (ACM)
- p. 1380-1391
Open AccessEnabling Agile Analysis of I/O Performance Data with PyDarshan
- 1 July 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in Computing in Science & Engineering
- Vol. 25 (4) , 35-41
Mochi: A Case Study in Translational Computer Science for High-Performance Computing Data Management
High-performance computing (HPC) has become an indispensable tool for solving diverse problems in science and engineering. Harnessing the power of HPC is not just a matter of efficient computation, however; it also calls for the efficient management of vast quantities of scientific data. This presents daunting challenges: rapidly evolving storage technology has motivated a shift toward increasingly complex, heterogeneous storage architectures that are difficult to optimize, and scientific data management needs have become every bit as diverse as the application domains that drive them. There is a clear need for agile, adaptable storage solutions that can be customized for the task and platform at hand. This motivated the establishment of the Mochi composable data service project. The Mochi project provides a library of robust, reusable, modular, and connectable data management components and microservices along with a methodology for composing them into specialized distributed data services. Mochi enables rapid deployment of custom data services with a high degree of developer productivity while still effectively leveraging cutting-edge HPC hardware. This article explores how the principles of translational computer science have been applied in practice in Mochi to achieve these goals.
- 1 July 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in Computing in Science & Engineering
- Vol. 25 (4) , 35-41
Mochi: A Case Study in Translational Computer Science for High-Performance Computing Data Management
High-performance computing (HPC) has become an indispensable tool for solving diverse problems in science and engineering. Harnessing the power of HPC is not just a matter of efficient computation, however; it also calls for the efficient management of vast quantities of scientific data. This presents daunting challenges: rapidly evolving storage technology has motivated a shift toward increasingly complex, heterogeneous storage architectures that are difficult to optimize, and scientific data management needs have become every bit as diverse as the application domains that drive them. There is a clear need for agile, adaptable storage solutions that can be customized for the task and platform at hand. This motivated the establishment of the Mochi composable data service project. The Mochi project provides a library of robust, reusable, modular, and connectable data management components and microservices along with a methodology for composing them into specialized distributed data services. Mochi enables rapid deployment of custom data services with a high degree of developer productivity while still effectively leveraging cutting-edge HPC hardware. This article explores how the principles of translational computer science have been applied in practice in Mochi to achieve these goals.
- 17 January 2020
- Published by Springer Nature in Journal of Computer Science and Technology
- Vol. 35 (1) , 121-144
Mochi: Composing Data Services for High-Performance Computing Environments
- 17 January 2020
- Published by Springer Nature in Journal of Computer Science and Technology
- Vol. 35 (1) , 121-144
Mochi: Composing Data Services for High-Performance Computing Environments