Arena: Efficiently Training Large Models via Dynamic Scheduling and Adaptive Parallelism Co-Design
Open Access
- 26 April 2026
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 89-107
Abstract
No abstract availableAll Related Versions
This publication has 21 references indexed in Scilit:
- Tenplex: Dynamic Parallelism for Deep Learning using Parallelizable Tensor Collections2024
- MAD-Max Beyond Single-Node: Enabling Large Machine Learning Model Acceleration on Distributed Systems2024
- Tessel: Boosting Distributed Execution of Large DNN Models via Flexible Schedule Search2024
- PyTorch FSDP: Experiences on Scaling Fully Sharded Data ParallelProceedings of the VLDB Endowment, 2023
- What Language Model to Train if You Have One Million GPU Hours?2022
- Efficient large-scale language model training on GPU clusters using megatron-LM2021
- ZeRO: Memory optimizations Toward Training Trillion Parameter Models2020
- Balancing efficiency and fairness in heterogeneous GPU clusters for deep learning2020
- PipeDream2019
- Scaling Distributed Machine Learning with the Parameter Server2014