Arnab Kumar Paul
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Research Subjects
Arnab Kumar Paul has a total of 66 co-authors. Here are the top co-authors by number of shared publications:
- Ali Raza Butt
Virginia Tech, Blacksburg, Virginia, USA
11 shared publications - Feiyi Wang
Oak Ridge National Laboratory
3 shared publications - Ahmad Maroof Karimi
National Center for Computational Sciences, Oak Ridge National Laboratory
2 shared publications - Ali Anwar
University of Minnesota
2 shared publications - Bibhudatta Sahoo
Department of Computer Science and Engineering, National Institute of Technology, Rourkela, India
2 shared publications - Dimitrios Skourtis
IBM Research - Almaden, San Jose, USA
2 shared publications
Latest Publications
- Open Access
- 13 July 2026
- Published by Association for Computing Machinery (ACM)
- p. 807-816
CoDL: A Framework for Studying Cross-Component Interference in Deep Learning Training Pipelines
- 13 July 2026
- Published by Association for Computing Machinery (ACM)
- p. 807-816
Open AccessCoDL: A Framework for Studying Cross-Component Interference in Deep Learning Training Pipelines
- Open Access
- 14 December 2025
- Published by Association for Computing Machinery (ACM)
- p. 285-297
UnifyFL: Enabling Decentralized Cross-Silo Federated Learning
- 14 December 2025
- Published by Association for Computing Machinery (ACM)
- p. 285-297
Open AccessUnifyFL: Enabling Decentralized Cross-Silo Federated Learning
- Open Access
- 4 January 2025
- Published by Association for Computing Machinery (ACM)
- p. 181-190
User-based I/O Profiling for Leadership Scale HPC Workloads
- 4 January 2025
- Published by Association for Computing Machinery (ACM)
- p. 181-190
Open AccessUser-based I/O Profiling for Leadership Scale HPC Workloads
- Open Access
- 20 November 2024
- Published by Association for Computing Machinery (ACM)
- p. 52-68
FedCaSe: Enhancing Federated Learning with Heterogeneity-aware Caching and Scheduling
Federated learning (FL) has emerged as a new paradigm of machine learning (ML) with the goal of collaborative learning on the vast pool of private data available across distributed edge devices. The focus of most existing works in FL systems has been on addressing the challenges of computation and communication heterogeneity inherent in training with edge devices. However, the crucial impact of I/O and the role of limited on-device storage has not been explored fully in FL context. Without policies to exploit the on-device storage for placement of client data samples, and schedule clients based on I/O benefits, FL training can lead to inefficiencies, such as increased training time and impacted accuracy convergence. In this paper, we propose FedCaSe, a framework for efficiently caching client samples in-situ on limited on-device storage and scheduling client participation. FedCaSe boosts the I/O performance by exploiting a unique characteristic--- the experience, i.e., relative impact on overall performance, of data samples and clients. FedCaSe utilizes this information in adaptive caching policies for sample placement inside the limited memory of edge clients. The framework also exploits the experience information to orchestrate the future selection of clients. Our experiments with representative workloads and policies show that compared to the state of the art, FedCaSe improves the training time by 2.06× for accuracy convergence at the scale of thousands of clients.
- 20 November 2024
- Published by Association for Computing Machinery (ACM)
- p. 52-68
Open AccessFedCaSe: Enhancing Federated Learning with Heterogeneity-aware Caching and Scheduling
Federated learning (FL) has emerged as a new paradigm of machine learning (ML) with the goal of collaborative learning on the vast pool of private data available across distributed edge devices. The focus of most existing works in FL systems has been on addressing the challenges of computation and communication heterogeneity inherent in training with edge devices. However, the crucial impact of I/O and the role of limited on-device storage has not been explored fully in FL context. Without policies to exploit the on-device storage for placement of client data samples, and schedule clients based on I/O benefits, FL training can lead to inefficiencies, such as increased training time and impacted accuracy convergence. In this paper, we propose FedCaSe, a framework for efficiently caching client samples in-situ on limited on-device storage and scheduling client participation. FedCaSe boosts the I/O performance by exploiting a unique characteristic--- the experience, i.e., relative impact on overall performance, of data samples and clients. FedCaSe utilizes this information in adaptive caching policies for sample placement inside the limited memory of edge clients. The framework also exploits the experience information to orchestrate the future selection of clients. Our experiments with representative workloads and policies show that compared to the state of the art, FedCaSe improves the training time by 2.06× for accuracy convergence at the scale of thousands of clients.
- Open Access
- 6 June 2024
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Storage
- Vol. 20 (3) , 1-35
An End-to-end High-performance Deduplication Scheme for Docker Registries and Docker Container Storage Systems
The wide adoption of Docker containers for supporting agile and elastic enterprise applications has led to a broad proliferation of container images. The associated storage performance and capacity requirements place a high pressure on the infrastructure of container registries that store and distribute images and container storage systems on the Docker client side that manage image layers and store ephemeral data generated at container runtime. The storage demand is worsened by the large amount of duplicate data in images. Moreover, container storage systems that use Copy-on-Write (CoW) file systems as storage drivers exacerbate the redundancy. Exploiting the high file redundancy in real-world images is a promising approach to drastically reduce the growing storage requirements of container registries and improve the space efficiency of container storage systems. However, existing deduplication techniques significantly degrade the performance of both registries and container storage systems because of data reconstruction overhead as well as the deduplication cost. We propose DupHunter, an end-to-end deduplication scheme that deduplicates layers for both Docker registries and container storage systems while maintaining a high image distribution speed and container I/O performance. DupHunter is divided into three tiers: registry tier, middle tier, and client tier. Specifically, we first build a high-performance deduplication engine at the registry tier that not only natively deduplicates layers for space savings but also reduces layer restore overhead. Then, we use deduplication offloading at the middle tier to eliminate the redundant files from the client tier and avoid bringing deduplication overhead to the clients. To further reduce the data duplicates caused by CoWs and improve the container I/O performance, we utilize a container-aware storage system at the client tier that reserves space for each container and arranges the placement of files and their modifications on the disk to preserve locality. Under real workloads, DupHunter reduces storage space by up to 6.9× and reduces the GET layer latency up to 2.8× compared to the state-of-the-art. Moreover, DupHunter can improve the container I/O performance by up to 93% for reads and 64% for writes.
- 6 June 2024
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Storage
- Vol. 20 (3) , 1-35
Open AccessAn End-to-end High-performance Deduplication Scheme for Docker Registries and Docker Container Storage Systems
The wide adoption of Docker containers for supporting agile and elastic enterprise applications has led to a broad proliferation of container images. The associated storage performance and capacity requirements place a high pressure on the infrastructure of container registries that store and distribute images and container storage systems on the Docker client side that manage image layers and store ephemeral data generated at container runtime. The storage demand is worsened by the large amount of duplicate data in images. Moreover, container storage systems that use Copy-on-Write (CoW) file systems as storage drivers exacerbate the redundancy. Exploiting the high file redundancy in real-world images is a promising approach to drastically reduce the growing storage requirements of container registries and improve the space efficiency of container storage systems. However, existing deduplication techniques significantly degrade the performance of both registries and container storage systems because of data reconstruction overhead as well as the deduplication cost. We propose DupHunter, an end-to-end deduplication scheme that deduplicates layers for both Docker registries and container storage systems while maintaining a high image distribution speed and container I/O performance. DupHunter is divided into three tiers: registry tier, middle tier, and client tier. Specifically, we first build a high-performance deduplication engine at the registry tier that not only natively deduplicates layers for space savings but also reduces layer restore overhead. Then, we use deduplication offloading at the middle tier to eliminate the redundant files from the client tier and avoid bringing deduplication overhead to the clients. To further reduce the data duplicates caused by CoWs and improve the container I/O performance, we utilize a container-aware storage system at the client tier that reserves space for each container and arranges the placement of files and their modifications on the disk to preserve locality. Under real workloads, DupHunter reduces storage space by up to 6.9× and reduces the GET layer latency up to 2.8× compared to the state-of-the-art. Moreover, DupHunter can improve the container I/O performance by up to 93% for reads and 64% for writes.
- Open Access
- 4 April 2024
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Storage
- Vol. 20 (2) , 1-42
Tarazu: An Adaptive End-to-end I/O Load-balancing Framework for Large-scale Parallel File Systems
The imbalanced I/O load on large parallel file systems affects the parallel I/O performance of high-performance computing (HPC) applications. One of the main reasons for I/O imbalances is the lack of a global view of system-wide resource consumption. While approaches to address the problem already exist, the diversity of HPC workloads combined with different file striping patterns prevents widespread adoption of these approaches. In addition, load-balancing techniques should be transparent to client applications. To address these issues, we propose Tarazu , an end-to-end control plane where clients transparently and adaptively write to a set of selected I/O servers to achieve balanced data placement. Our control plane leverages real-time load statistics for global data placement on distributed storage servers, while our design model employs trace-based optimization techniques to minimize latency for I/O load requests between clients and servers and to handle multiple striping patterns in files. We evaluate our proposed system on an experimental cluster for two common use cases: the synthetic I/O benchmark IOR and the scientific application I/O kernel HACC-I/O. We also use a discrete-time simulator with real HPC application traces from emerging workloads running on the Summit supercomputer to validate the effectiveness and scalability of Tarazu in large-scale storage environments. The results show improvements in load balancing and read performance of up to 33% and 43%, respectively, compared to the state-of-the-art.
- 4 April 2024
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Storage
- Vol. 20 (2) , 1-42
Open AccessTarazu: An Adaptive End-to-end I/O Load-balancing Framework for Large-scale Parallel File Systems
The imbalanced I/O load on large parallel file systems affects the parallel I/O performance of high-performance computing (HPC) applications. One of the main reasons for I/O imbalances is the lack of a global view of system-wide resource consumption. While approaches to address the problem already exist, the diversity of HPC workloads combined with different file striping patterns prevents widespread adoption of these approaches. In addition, load-balancing techniques should be transparent to client applications. To address these issues, we propose Tarazu , an end-to-end control plane where clients transparently and adaptively write to a set of selected I/O servers to achieve balanced data placement. Our control plane leverages real-time load statistics for global data placement on distributed storage servers, while our design model employs trace-based optimization techniques to minimize latency for I/O load requests between clients and servers and to handle multiple striping patterns in files. We evaluate our proposed system on an experimental cluster for two common use cases: the synthetic I/O benchmark IOR and the scientific application I/O kernel HACC-I/O. We also use a discrete-time simulator with real HPC application traces from emerging workloads running on the Summit supercomputer to validate the effectiveness and scalability of Tarazu in large-scale storage environments. The results show improvements in load balancing and read performance of up to 33% and 43%, respectively, compared to the state-of-the-art.
- 16 October 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 1-4
Analyzing File Access Patterns on Large-Scale HPC Systems: Opportunities for File Prefetching
This paper explores the potential opportunities for implementing file prefetching techniques on large-scale high-performance computing (HPC) systems. Specifically, we investigate the file access patterns of various applications across multiple scientific domains using two years' worth of Darshan I/O traces obtained from the Summit supercomputer. We identify recurring trends and patterns which indicate that prefetching can be effectively leveraged to improve data access performance on HPC systems. This study serves as a valuable reference for system architects and developers in the HPC community, providing insights into the opportunities and challenges associated with enabling file prefetching on large-scale HPC systems.
- 16 October 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 1-4
Analyzing File Access Patterns on Large-Scale HPC Systems: Opportunities for File Prefetching
This paper explores the potential opportunities for implementing file prefetching techniques on large-scale high-performance computing (HPC) systems. Specifically, we investigate the file access patterns of various applications across multiple scientific domains using two years' worth of Darshan I/O traces obtained from the Summit supercomputer. We identify recurring trends and patterns which indicate that prefetching can be effectively leveraged to improve data access performance on HPC systems. This study serves as a valuable reference for system architects and developers in the HPC community, providing insights into the opportunities and challenges associated with enabling file prefetching on large-scale HPC systems.
- Open Access
- 4 January 2023
- Published by Association for Computing Machinery (ACM)
- p. 282-283
A Data-Centric Approach for Analyzing Large-Scale Deep Learning Applications
Over the past two decades, deep learning techniques have emerged as an immensely powerful technology, with break-through in computer vision, speech to text technologies, natural language processing and many more such fields. As neural networks grow in size and capacity, they require higher compute power and larger datasets to converge to a model with higher accuracy. In this poster, we present a data-centric approach to studying the system level requirements (GPU utilization, CPU utilization, I/O) of deep learning training workloads, and uncover a few insights that help us understand the nature of deep learning training workloads. We analyse three datasets found in industry, academia and national laboratories to understand the requirements and properties of deep learning training workloads.
- 4 January 2023
- Published by Association for Computing Machinery (ACM)
- p. 282-283
Open AccessA Data-Centric Approach for Analyzing Large-Scale Deep Learning Applications
Over the past two decades, deep learning techniques have emerged as an immensely powerful technology, with break-through in computer vision, speech to text technologies, natural language processing and many more such fields. As neural networks grow in size and capacity, they require higher compute power and larger datasets to converge to a model with higher accuracy. In this poster, we present a data-centric approach to studying the system level requirements (GPU utilization, CPU utilization, I/O) of deep learning training workloads, and uncover a few insights that help us understand the nature of deep learning training workloads. We analyse three datasets found in industry, academia and national laboratories to understand the requirements and properties of deep learning training workloads.
- Open Access
- 4 January 2023
- Published by Association for Computing Machinery (ACM)
- p. 318-323
Characteristics of Deep Learning Workloads in Industry, Academic Institutions and National Laboratories
Over the past two decades, deep learning techniques have emerged as an immensely powerful technology, with break-through in computer vision, speech to text technologies, natural language processing and many more such fields. As neural networks grow in size and capacity, they require higher compute power and larger datasets to converge to a model with higher accuracy. In this paper, we present a data-centric approach to study the system level requirements (GPU utilization, CPU utilization, I/O) of deep learning training workloads, and uncover a few insights that help us understand the nature of deep learning training workloads. We analyze three datasets found in industry, academia and national laboratories to understand the requirements and properties of deep learning training workloads in different settings.
- 4 January 2023
- Published by Association for Computing Machinery (ACM)
- p. 318-323
Open AccessCharacteristics of Deep Learning Workloads in Industry, Academic Institutions and National Laboratories
Over the past two decades, deep learning techniques have emerged as an immensely powerful technology, with break-through in computer vision, speech to text technologies, natural language processing and many more such fields. As neural networks grow in size and capacity, they require higher compute power and larger datasets to converge to a model with higher accuracy. In this paper, we present a data-centric approach to study the system level requirements (GPU utilization, CPU utilization, I/O) of deep learning training workloads, and uncover a few insights that help us understand the nature of deep learning training workloads. We analyze three datasets found in industry, academia and national laboratories to understand the requirements and properties of deep learning training workloads in different settings.
- 1 December 2020
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 202-211
Understanding HPC Application I/O Behavior Using System Level Statistics
The processor performance of high performance computing (HPC) systems is increasing at a much higher rate than storage performance. This imbalance leads to I/O performance bottlenecks in massively parallel HPC applications. Therefore, there is a need for improvements in storage and file system designs to meet the ever-growing I/O needs of HPC applications. Storage and file system designers require a deep understanding of how HPC application I/O behavior affects current storage system installations in order to improve them. In this work, we contribute to this understanding using application-agnostic file system statistics gathered on compute nodes as well as metadata and object storage file system servers. We analyze file system statistics of more than 4 million jobs over a period of three years on two systems at Lawrence Livermore National Laboratory that include a 15 PiB Lustre file system for storage. The results of our study add to the state-of-the-art in I/O understanding by providing insight into how general HPC workloads affect the performance of large-scale storage systems. Some key observations in our study show that reads and writes are evenly distributed across the storage system; applications which perform I/O, spread that I/O across ~78% of the minutes of their runtime on average; less than 22% of HPC users who submit write-intensive jobs perform efficient writes to the file system; and I/O contention seriously impacts I/O performance.
- 1 December 2020
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 202-211
Understanding HPC Application I/O Behavior Using System Level Statistics
The processor performance of high performance computing (HPC) systems is increasing at a much higher rate than storage performance. This imbalance leads to I/O performance bottlenecks in massively parallel HPC applications. Therefore, there is a need for improvements in storage and file system designs to meet the ever-growing I/O needs of HPC applications. Storage and file system designers require a deep understanding of how HPC application I/O behavior affects current storage system installations in order to improve them. In this work, we contribute to this understanding using application-agnostic file system statistics gathered on compute nodes as well as metadata and object storage file system servers. We analyze file system statistics of more than 4 million jobs over a period of three years on two systems at Lawrence Livermore National Laboratory that include a 15 PiB Lustre file system for storage. The results of our study add to the state-of-the-art in I/O understanding by providing insight into how general HPC workloads affect the performance of large-scale storage systems. Some key observations in our study show that reads and writes are evenly distributed across the storage system; applications which perform I/O, spread that I/O across ~78% of the minutes of their runtime on average; less than 22% of HPC users who submit write-intensive jobs perform efficient writes to the file system; and I/O contention seriously impacts I/O performance.