Single-model uncertainty quantification in neural network potentials does not consistently outperform model ensembles
Top Cited Papers
Open Access
- 16 December 2023
- journal article
- research article
- Published by Springer Nature in npj Computational Materials
- Vol. 9 (1) , 1-11
Abstract
Neural networks (NNs) often assign high confidence to their predictions, even for points far out of distribution, making uncertainty quantification (UQ) a challenge. When they are employed to model interatomic potentials in materials systems, this problem leads to unphysical structures that disrupt simulations, or to biased statistics and dynamics that do not reflect the true physics. Differentiable UQ techniques can find new informative data and drive active learning loops for robust potentials. However, a variety of UQ techniques, including newly developed ones, exist for atomistic simulations and there are no clear guidelines for which are most effective or suitable for a given case. In this work, we examine multiple UQ schemes for improving the robustness of NN interatomic potentials (NNIPs) through active learning. In particular, we compare incumbent ensemble-based methods against strategies that use single, deterministic NNs: mean-variance estimation (MVE), deep evidential regression, and Gaussian mixture models (GMM). We explore three datasets ranging from in-domain interpolative learning to more extrapolative out-of-domain generalization challenges: rMD17, ammonia inversion, and bulk silica glass. Performance is measured across multiple metrics relating model error to uncertainty. Our experiments show that none of the methods consistently outperformed each other across the various metrics. Ensembling remained better at generalization and for NNIP robustness; MVE only proved effective for in-domain interpolation, while GMM was better out-of-domain; and evidential regression, despite its promise, was not the preferable alternative in any of the cases. More broadly, cost-effective, single deterministic models cannot yet consistently match or outperform ensembling for uncertainty quantification in NNIPs.Keywords
All Related Versions
Funding Information
- AGC, Inc.
- AGC, Inc.
This publication has 65 references indexed in Scilit:
- Ensemble approaches for regressionACM Computing Surveys, 2012
- Generalized Neural-Network Representation of High-Dimensional Potential-Energy SurfacesPhysical Review Letters, 2007
- An ab initio parametrized interatomic force field for silicaThe Journal of Chemical Physics, 2002
- From ultrasoft pseudopotentials to the projector augmented-wave methodPhysical Review B, 1999
- Error Correlation and Error Reduction in Ensemble ClassifiersConnection Science, 1996
- Generalized Gradient Approximation Made SimplePhysical Review Letters, 1996
- Efficient iterative schemes for ab initio total-energy calculations using a plane-wave basis setPhysical Review B, 1996
- Fast Parallel Algorithms for Short-Range Molecular DynamicsJournal of Computational Physics, 1995
- Force fields for silicas and aluminophosphates based onab initiocalculationsPhysical Review Letters, 1990
- First-Principles Interatomic Potential of Silica Applied to Molecular DynamicsPhysical Review Letters, 1988