NDPipe: Exploiting Near-data Processing for Scalable Inference and Continuous Training in Photo Storage
Open Access
- 27 April 2024
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 689-707
Abstract
This paper proposes a novel photo storage system called NDPipe, which accelerates the performance of training and inference for image data by leveraging near-data processing in photo storage servers. NDPipe distributes storage servers with inexpensive commodity GPUs in a data center and uses their collective intelligence to perform inference and training near image data. By efficiently partitioning deep neural network (DNN) models and exploiting the data parallelism of many storage servers, NDPipe can achieve high training throughput with low synchronization costs. NDPipe optimizes the near-data processing engine to maximally utilize system components in each storage server. Our results show that, given the same energy budget, NDPipe exhibits 1.39× higher inference throughput and 2.64× faster training speed than typical photo storage systems.Keywords
This publication has 41 references indexed in Scilit:
- ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devices2018
- Aggregated Residual Transformations for Deep Neural Networks2017
- Overcoming catastrophic forgetting in neural networksProceedings of the National Academy of Sciences, 2017
- Deep Residual Learning for Image Recognition2016
- Rethinking the Inception Architecture for Computer Vision2016
- ImageNet Large Scale Visual Recognition ChallengeInternational Journal of Computer Vision, 2015
- ImageNet: A large-scale hierarchical image database2009
- Bandwidth optimal all-reduce algorithms for clusters of workstationsJournal of Parallel and Distributed Computing, 2009
- Learning with Drift Detection2004
- Probability Inequalities for Sums of Bounded Random VariablesJournal of the American Statistical Association, 1963