Dong Dai
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Research Subjects
Dong Dai has a total of 85 co-authors. Here are the top co-authors by number of shared publications:
- Yong Chen
Department of Computer Science, Texas Tech University, Lubbock, TX, USA
11 shared publications - Robert Ross
Argonne National Laboratory, Lemont, USA
6 shared publications - Philip Carns
Argonne National Laboratory (ANL), Lemont, USA
5 shared publications - Chris Egersdoerfer
University of North Carolina at Charlotte, Charlotte, NC, USA
3 shared publications - Abdullah Al Raqibul Islam
University of North Carolina at Charlotte, Charlotte, USA
2 shared publications - Di Zhang
University of North Carolina at Charlotte, United States of America
2 shared publications
Latest Publications
- Open Access
- 15 November 2025
- Published by Association for Computing Machinery (ACM)
- p. 1366-1374
LLMTailor: A Layer-wise Tailoring Tool for Efficient Checkpointing of Large Language Models
- 15 November 2025
- Published by Association for Computing Machinery (ACM)
- p. 1366-1374
Open AccessLLMTailor: A Layer-wise Tailoring Tool for Efficient Checkpointing of Large Language Models
- Open Access
- 15 November 2025
- Published by Association for Computing Machinery (ACM)
- p. 1406-1414
RL4Sys: A Lightweight System-driven RL Framework for Drop-in Integration in System Optimization
- 15 November 2025
- Published by Association for Computing Machinery (ACM)
- p. 1406-1414
Open AccessRL4Sys: A Lightweight System-driven RL Framework for Drop-in Integration in System Optimization
- Open Access
- 15 November 2025
- Published by Association for Computing Machinery (ACM)
- p. 1446-1463
Improving SpGEMM Performance Through Matrix-Reordering and Cluster-wise Computation
- 15 November 2025
- Published by Association for Computing Machinery (ACM)
- p. 1446-1463
Open AccessImproving SpGEMM Performance Through Matrix-Reordering and Cluster-wise Computation
- Open Access
- 15 November 2025
- Published by Association for Computing Machinery (ACM)
- p. 1252-1266
STELLAR: Storage Tuning Engine Leveraging LLM Autonomous Reasoning for High Performance Parallel File Systems
- 15 November 2025
- Published by Association for Computing Machinery (ACM)
- p. 1252-1266
Open AccessSTELLAR: Storage Tuning Engine Leveraging LLM Autonomous Reasoning for High Performance Parallel File Systems
- Open Access
- 20 July 2025
- Published by Association for Computing Machinery (ACM)
- p. 1-14
TSUE: A Two-Stage Data Update Method for an Erasure Coded Cluster File System
- 20 July 2025
- Published by Association for Computing Machinery (ACM)
- p. 1-14
Open AccessTSUE: A Two-Stage Data Update Method for an Erasure Coded Cluster File System
- Open Access
- 4 May 2025
- Published by Association for Computing Machinery (ACM)
- p. 79-92
Towards Empirical Roofline Modeling of Distributed Data Services: Mapping the Boundaries of RPC Throughput
- 4 May 2025
- Published by Association for Computing Machinery (ACM)
- p. 79-92
Open AccessTowards Empirical Roofline Modeling of Distributed Data Services: Mapping the Boundaries of RPC Throughput
- 16 December 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Computers
- Vol. 74  (4) , 1224-1238
Hardware Accelerated Vision Transformer via Heterogeneous Architecture Design and Adaptive Dataflow Mapping
Vision transformer (ViT) models have demonstrated remarkable advantages in visual tasks. However, the ViT model contains various types of operators, and its sophisticated model structure imposes substantial computational complexity and storage burden. Existing hardware solutions still fail to fully unleash the ViT acceleration potential due to the mismatch between operators and hardware architectures, suffering from inefficient dataflow mapping. This work proposes HDViT, a full-fledged heterogeneous hardware accelerator on FPGA, to enhance the ViT acceleration by comprehensively analyzing and addressing the challenges of heterogeneous architecture design. Specifically, HDViT first develops a heterogeneous architecture design that is composed of multiple processing engines (PEs) to accelerate various operators in the ViT model. Then, HDViT devises a hybrid-oriented dataflow mapping strategy to reduce data transmission granularity and alleviate storage resource pressure. Lastly, to achieve the latency balancing among multiple PEs, we formulate the HDViT architecture and implement an automated exploration process to identify optimized parallelism parameters that satisfy computation and storage demands while enhancing the heterogeneous architectural performance. Experimental results indicate that HDViT achieves significant performance speedups of 2.16$\times$ and 3.51$\times$ compared to previous heterogeneous and unified accelerators, respectively. HDViT also achieves a maximum of 98.46% hardware utilization.- 16 December 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Computers
- Vol. 74  (4) , 1224-1238
Hardware Accelerated Vision Transformer via Heterogeneous Architecture Design and Adaptive Dataflow Mapping
Vision transformer (ViT) models have demonstrated remarkable advantages in visual tasks. However, the ViT model contains various types of operators, and its sophisticated model structure imposes substantial computational complexity and storage burden. Existing hardware solutions still fail to fully unleash the ViT acceleration potential due to the mismatch between operators and hardware architectures, suffering from inefficient dataflow mapping. This work proposes HDViT, a full-fledged heterogeneous hardware accelerator on FPGA, to enhance the ViT acceleration by comprehensively analyzing and addressing the challenges of heterogeneous architecture design. Specifically, HDViT first develops a heterogeneous architecture design that is composed of multiple processing engines (PEs) to accelerate various operators in the ViT model. Then, HDViT devises a hybrid-oriented dataflow mapping strategy to reduce data transmission granularity and alleviate storage resource pressure. Lastly, to achieve the latency balancing among multiple PEs, we formulate the HDViT architecture and implement an automated exploration process to identify optimized parallelism parameters that satisfy computation and storage demands while enhancing the heterogeneous architectural performance. Experimental results indicate that HDViT achieves significant performance speedups of 2.16$\times$ and 3.51$\times$ compared to previous heterogeneous and unified accelerators, respectively. HDViT also achieves a maximum of 98.46% hardware utilization.- Open Access
- 8 July 2024
- Published by Association for Computing Machinery (ACM)
- p. 86-92
ION: Navigating the HPC I/O Optimization Journey using Large Language Models
Effectively leveraging the complex software and hardware I/O stacks of HPC systems to deliver needed I/O performance has been a challenging task for domain scientists. To identify and address I/O issues in their applications, scientists largely rely on I/O experts to analyze the recorded I/O traces of their applications and provide insights into the potential issues. However, due to the limited number of I/O experts and the growing demand for data-intensive applications across the wide spectrum of sciences, inaccessibility has become a major bottleneck hindering scientists from maximizing their productivity. Inspired by the recent rapid progress of large language models (LLMs), in this work we propose IO Navigator (ION), an LLM-based framework that takes a recorded I/O trace of an application as input and leverages the in-context learning, chain-of-thought, and code generation capabilities of LLMs to comprehensively analyze the I/O trace and provide diagnosis of potential I/O issues. Similar to an I/O expert, ION provides detailed justifications for the diagnosis and an interactive interface for scientists to ask detailed questions about the diagnosis. We illustrate ION's applicability by assessing it on a set of controlled I/O traces generated with different I/O issues. We also demonstrate that ION can match state-of-the-art I/O optimization tools and provide more insightful and adaptive diagnoses for real applications. We believe ION, with its full capabilities, has the potential to become a powerful tool for scientists to navigate through complex I/O subsystems in the future.
- 8 July 2024
- Published by Association for Computing Machinery (ACM)
- p. 86-92
Open AccessION: Navigating the HPC I/O Optimization Journey using Large Language Models
Effectively leveraging the complex software and hardware I/O stacks of HPC systems to deliver needed I/O performance has been a challenging task for domain scientists. To identify and address I/O issues in their applications, scientists largely rely on I/O experts to analyze the recorded I/O traces of their applications and provide insights into the potential issues. However, due to the limited number of I/O experts and the growing demand for data-intensive applications across the wide spectrum of sciences, inaccessibility has become a major bottleneck hindering scientists from maximizing their productivity. Inspired by the recent rapid progress of large language models (LLMs), in this work we propose IO Navigator (ION), an LLM-based framework that takes a recorded I/O trace of an application as input and leverages the in-context learning, chain-of-thought, and code generation capabilities of LLMs to comprehensively analyze the I/O trace and provide diagnosis of potential I/O issues. Similar to an I/O expert, ION provides detailed justifications for the diagnosis and an interactive interface for scientists to ask detailed questions about the diagnosis. We illustrate ION's applicability by assessing it on a set of controlled I/O traces generated with different I/O issues. We also demonstrate that ION can match state-of-the-art I/O optimization tools and provide more insightful and adaptive diagnoses for real applications. We believe ION, with its full capabilities, has the potential to become a powerful tool for scientists to navigate through complex I/O subsystems in the future.
- 14 March 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Parallel and Distributed Systems
- Vol. 35  (5) , 844-861
PROV-IO: A Cross-Platform Provenance Framework for Scientific Data on HPC Systems
Data provenance, or data lineage, describes the life cycle of data. In scientific workflows on HPC systems, scientists often seek diverse provenance (e.g., origins of data products, usage patterns of datasets). Unfortunately, existing provenance solutions cannot address the challenges due to their incompatible provenance models and/or system implementations. In this paper, we analyze four representative scientific workflows in collaboration with the domain scientists to identify concrete provenance needs. Based on the first-hand analysis, we propose a provenance framework called PROV-IO$^+$ , which includes an I/O-centric provenance model for describing scientific data and the associated I/O operations and environments precisely. Moreover, we build a prototype of PROV-IO$^+$ to enable end-to-end provenance support on real HPC systems with little manual effort. The PROV-IO$^+$ framework can support both containerized and non-containerized workflows on different HPC platforms with flexibility in selecting various classes of provenance. Our experiments with realistic workflows show that PROV-IO$^+$ can address the provenance needs of the domain scientists effectively with reasonable performance (e.g., less than 3.5% tracking overhead for most experiments). Moreover, PROV-IO$^+$ outperforms a state-of-the-art system (i.e., ProvLake) in our experiments.- 14 March 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Parallel and Distributed Systems
- Vol. 35  (5) , 844-861
PROV-IO: A Cross-Platform Provenance Framework for Scientific Data on HPC Systems
Data provenance, or data lineage, describes the life cycle of data. In scientific workflows on HPC systems, scientists often seek diverse provenance (e.g., origins of data products, usage patterns of datasets). Unfortunately, existing provenance solutions cannot address the challenges due to their incompatible provenance models and/or system implementations. In this paper, we analyze four representative scientific workflows in collaboration with the domain scientists to identify concrete provenance needs. Based on the first-hand analysis, we propose a provenance framework called PROV-IO$^+$ , which includes an I/O-centric provenance model for describing scientific data and the associated I/O operations and environments precisely. Moreover, we build a prototype of PROV-IO$^+$ to enable end-to-end provenance support on real HPC systems with little manual effort. The PROV-IO$^+$ framework can support both containerized and non-containerized workflows on different HPC platforms with flexibility in selecting various classes of provenance. Our experiments with realistic workflows show that PROV-IO$^+$ can address the provenance needs of the domain scientists effectively with reasonable performance (e.g., less than 3.5% tracking overhead for most experiments). Moreover, PROV-IO$^+$ outperforms a state-of-the-art system (i.e., ProvLake) in our experiments.- Open Access
- 12 November 2023
- Published by Association for Computing Machinery (ACM)
- p. 1316-1323
A Reinforcement Learning Based Backfilling Strategy for HPC Batch Jobs
High Performance Computing (HPC) systems are used across a wide range of disciplines for both large and complex computations. HPC systems often receive many thousands of computational tasks at a time, colloquially referred to as “jobs”. These jobs must then be scheduled as optimally as possible so they can be completed within a reasonable timeframe. HPC scheduling systems often employ a technique called “backfilling”, wherein low-priority jobs are scheduled earlier to use the available resources that are waiting for the pending high-priority jobs. To make it work, backfilling largely relies on job runtime to calculate the start time of the ready-to-schedule jobs and avoid delaying them. It is a common belief that better estimations of job runtime will lead to better backfilling and more effective scheduling. However, our experiments show a different conclusion: there is a missing trade-off between prediction accuracy and backfilling opportunities. To learn how to achieve the best trade-off, we believe reinforcement learning (RL) can be effectively leveraged. Reinforcement Learning relies on an “agent” which makes decisions from observing the environment, and gains rewards or punishments based on the quality of its decision-making. Based on this idea, we designed RLBackfilling, a reinforcement learning-based backfilling algorithm. We show how RLBackfilling can learn effective backfilling strategies via trial-and-error on existing job traces. Our evaluation results show up to 17x better scheduling performance (based on average bounded job slowdown) compared to EASY backfilling using user-provided job runtime and 4.7x better performance compared with EASY using the ideal predicted job runtime (the actual job runtime).
- 12 November 2023
- Published by Association for Computing Machinery (ACM)
- p. 1316-1323
Open AccessA Reinforcement Learning Based Backfilling Strategy for HPC Batch Jobs
High Performance Computing (HPC) systems are used across a wide range of disciplines for both large and complex computations. HPC systems often receive many thousands of computational tasks at a time, colloquially referred to as “jobs”. These jobs must then be scheduled as optimally as possible so they can be completed within a reasonable timeframe. HPC scheduling systems often employ a technique called “backfilling”, wherein low-priority jobs are scheduled earlier to use the available resources that are waiting for the pending high-priority jobs. To make it work, backfilling largely relies on job runtime to calculate the start time of the ready-to-schedule jobs and avoid delaying them. It is a common belief that better estimations of job runtime will lead to better backfilling and more effective scheduling. However, our experiments show a different conclusion: there is a missing trade-off between prediction accuracy and backfilling opportunities. To learn how to achieve the best trade-off, we believe reinforcement learning (RL) can be effectively leveraged. Reinforcement Learning relies on an “agent” which makes decisions from observing the environment, and gains rewards or punishments based on the quality of its decision-making. Based on this idea, we designed RLBackfilling, a reinforcement learning-based backfilling algorithm. We show how RLBackfilling can learn effective backfilling strategies via trial-and-error on existing job traces. Our evaluation results show up to 17x better scheduling performance (based on average bounded job slowdown) compared to EASY backfilling using user-provided job runtime and 4.7x better performance compared with EASY using the ideal predicted job runtime (the actual job runtime).