Open Access
- 22 April 2024
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 65-72
Abstract
Data preprocessing consisting of tasks like sample resizing, cropping, and filtering, is a crucial step in machine learning (ML) workflows. Even though the preprocessing step is largely ignored by work that focuses on optimizing training algorithms, in practice for many workloads preprocessing and training are pipelined. Popular ML frameworks like PyTorch use data loaders to feed data into model training. If the pipeline between preprocessing and training is not done carefully, it can cause significant waiting times on the GPU side. To address this limitation, we introduce SpeedyLoader, a system that overlaps preprocessing and training by leveraging asynchronous data preprocessing and avoiding head-of-line blocking. SpeedyLoader incorporates dedicated data loading threads, which organize preprocessed samples into queues based on their predicted processing times. Concurrently, GPUs fetch samples from these queues, ensuring training is not impeded by preprocessing completion. Compared to the default PyTorch DataLoader, SpeedyLoader reduces training time by up to 30% and increases GPU usage by 4.3×, all while maintaining a consistent evaluation accuracy of 91%.Keywords
This publication has 6 references indexed in Scilit:
- Orion: Interference-aware, Fine-grained GPU Sharing for ML Applications2024
- A joint management middleware to improve training performance of deep recommendation systems with SSDs2022
- Understanding data storage and ingestion for large-scale deep recommendation model training2022
- Chasing Carbon: The Elusive Environmental Footprint of Computing2021
- Analysis of Dimensionality Reduction Techniques on Big DataIEEE Access, 2020
- 3D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotation2016