Towards Scalable Storage Architectures for GPU Clusters Running Large Language Models
Open Access
- 20 July 2026
- journal article
- Published by Association for Computing Machinery (ACM) in ACM SIGMETRICS Performance Evaluation Review
- Vol. 54 (1) , 123-125
Abstract
Accelerators in multi-GPU nodes increasingly sit idle due to storage and I/O inefficiencies. We present a phase-aware evaluation of storage datapaths for Large Language Models (LLMs) on single-node, multi-GPU systems, comparing libaio, io_uring, Storage Performance Development Kit (SPDK), and GPUDirect Storage (GDS) across SATA SSDs, NVMe SSDs, Optane NVMe, and Optane Persistent Memory. Using an automated framework, we test over 25,000 configurations and measure throughput, latency, Input Output Per Second (IOPS), and CPU overhead using workloads derived from real LLM traces spanning pre-training, fine-tuning, and inference. For inference workloads dominated by small random I/O, io_uring provides the best latency and competitive IOPS on NVMe devices. For pre-training and fine-tuning, which rely on large sequential transfers, GDS significantly reduces model load times and CPU usage. These findings show that optimal storage datapaths are phase-dependent and offer practical guidance for narrowing the storage--compute gap in GPU-based LLM systems.Keywords
This publication has 4 references indexed in Scilit:
- Towards Scalable Storage Architectures for GPU Clusters Running Large Language ModelsProceedings of the ACM on Measurement and Analysis of Computing Systems, 2026
- Performance Characterization of Modern Storage Stacks: POSIX I/O, libaio, SPDK, and io_uring2023
- Analyzing and mitigating data stalls in DNN trainingProceedings of the VLDB Endowment, 2021
- SPDK: A Development Kit to Build High Performance Storage Applications2017