Alexander Weber
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Research Interests
Alexander Weber has a total of 10 co-authors. Here are the top co-authors by number of shared publications:
- David Fitzek1 shared publication
- Felix Stollenwerk
AI Sweden
1 shared publication - Joachim Köhler
NetMedia Fraunhofer IAIS Sankt Augustin Germany
1 shared publication - Max Lübbering
Fraunhofer IAIS
1 shared publication - Mehdi Ali
Fraunhofer IAIS Germany
1 shared publication - Michael Fromm
Fraunhofer IAIS Germany
1 shared publication
Latest Publications
- Open Access
Modalities, a PyTorch-native Framework For Large-scale LLM Training and Research
Today's LLM (pre-) training and research workflows typically allocate a significant amount of compute to large-scale ablation studies. Despite the substantial compute costs of these ablations, existing open-source frameworks provide limited tooling for these experiments, often forcing researchers to write their own wrappers and scripts. We propose Modalities, an end-to-end PyTorch-native framework that integrates data-driven LLM research with large-scale model training from two angles. Firstly, by integrating state-of-the-art parallelization strategies, it enables both efficient pretraining and systematic ablations at trillion-token and billion-parameter scale. Secondly, Modalities adopts modular design with declarative, self-contained configuration, enabling reproducibility and extensibility levels that are difficult to achieve out-of-the-box with existing LLM training frameworks.
Open AccessModalities, a PyTorch-native Framework For Large-scale LLM Training and Research
Today's LLM (pre-) training and research workflows typically allocate a significant amount of compute to large-scale ablation studies. Despite the substantial compute costs of these ablations, existing open-source frameworks provide limited tooling for these experiments, often forcing researchers to write their own wrappers and scripts. We propose Modalities, an end-to-end PyTorch-native framework that integrates data-driven LLM research with large-scale model training from two angles. Firstly, by integrating state-of-the-art parallelization strategies, it enables both efficient pretraining and systematic ablations at trillion-token and billion-parameter scale. Secondly, Modalities adopts modular design with declarative, self-contained configuration, enabling reproducibility and extensibility levels that are difficult to achieve out-of-the-box with existing LLM training frameworks.