The Case for Strong Scaling in Deep Learning: Training Large 3D CNNs with Hybrid Parallelism
Authors:
- Yosuke OyamaYOYosuke Oyama
- Naoya MaruyamaNMNaoya Maruyama
- Nikoli DrydenNDNikoli Dryden
- Erin McCarthyEMErin McCarthy
- Peter HarringtonPHPeter Harrington
- Jan BalewskiJBJan Balewski
- Satoshi MatsuokaSMSatoshi Matsuoka
- Peter NugentPNPeter Nugent
- Brian Van EssenBEBrian Van Essen
Preprint