Understanding multi-fidelity training of machine-learned force-fields
Open Access
- 1 August 2026
- journal article
- Published by IOP Publishing in Machine Learning: Science and Technology
- Vol. 7 (4) , 045057
Abstract
This study systematically investigates two multi-fidelity strategies used to train machine-learned force fields (MLFFs)—pre-training/fine-tuning and multi-headed training—and elucidates the mechanisms underpinning their success. For pre-training and fine-tuning, we uncover a log–log linear relationship between pre-trained and fine-tuned accuracies that holds across model architectures, model sizes, and quantum-chemical methods. The success of this approach hinges on the quantity and quality of available pre-training data, and, critically, the inclusion of force labels. We demonstrate that pre-trained representations are inherently method-specific, requiring adaptation of the model backbone during fine-tuning. In contrast, multi-headed models learn method-independent backbone representations, where again the heads’ accuracies are log–log linearly related. Relative to pre-training and fine-tuning, these shared representations marginally reduce model performance in most cases. However, this trade-off is offset by practical advantages: multi-headed training extends naturally to multiple labelling methods and enables partial replacement of expensive labels with cheaper alternatives, paving the way towards cost-efficient universal MLFFs.Keywords
All Related Versions
This publication has 63 references indexed in Scilit:
- Towards Routine Condensed Phase Simulations with Delta-Learned Coupled Cluster Accuracy: Application to Liquid WaterJournal of Chemical Theory and Computation, 2025
- One to Rule Them All: A Universal Interatomic Potential Learning across Quantum Chemical LevelsJournal of Chemical Theory and Computation, 2025
- The emergence of machine learning force fields in drug designMedicinal Research Reviews, 2024
- Data-Efficient Machine Learning Potentials from Transfer Learning of Periodic Correlated Electronic Structure Methods: Liquid Water at AFQMC, CCSD, and CCSD(T) AccuracyJournal of Chemical Theory and Computation, 2023
- Four Generations of High-Dimensional Neural Network PotentialsChemical Reviews, 2021
- Machine Learning Force FieldsChemical Reviews, 2021
- Extended tight‐binding quantum chemistry methodsWIREs Computational Molecular Science, 2020
- Machine Learning Interatomic Potentials as Emerging Tools for Materials ScienceAdvanced Materials, 2019
- Multi-fidelity machine learning models for accurate bandgap predictions of solidsComputational Materials Science, 2017
- Big Data Meets Quantum Chemistry Approximations: The Δ-Machine Learning ApproachJournal of Chemical Theory and Computation, 2015