Transfer Learning for a Foundational Chemistry Model
Preprint
- 20 September 2023
- preprint
- Published by American Chemical Society (ACS) in ChemRxiv
Abstract
Data-driven chemistry has garnered much interest concurrent with improvements in hardware and the development of new machine learning models. However, a notable bottleneck for data-driven chemistry specifically is the challenge in obtaining sufficiently large, accurate datasets of a desired chemical outcome. Herein, I develop a machine learning framework that makes prediction amid low data: First, a chemical “foundational model” is trained using on a dataset of ~1 million experimental organic crystal structures of organic molecules - a source of big data in the chemistry. A task specific model is then stacked on top on this general model. This approach achieves state-of-the-art performance in a diverse set of tasks – toxicity prediction, yield prediction, and odor prediction. More generally, my work shows that a foundational model approach, which led to step-change in domains such as natural language, can unlock advances in chemistry.Keywords
All Related Versions
This publication has 29 references indexed in Scilit:
- Gaussian Process Regression Models for Predicting Atomic Energies and Multipole MomentsJournal of Chemical Theory and Computation, 2023
- Machine Learning for Chemical Reactivity: The Importance of Failed ExperimentsAngewandte Chemie International Edition, 2022
- Predicting Infrared Spectra with Message Passing Neural NetworksJournal of Chemical Information and Modeling, 2021
- Predicting Energetics Materials’ Crystalline Density from Chemical Structure by Machine LearningJournal of Chemical Information and Modeling, 2021
- Applications of Deep Learning in Molecule Generation and Molecular Property PredictionAccounts of Chemical Research, 2020
- Deep Learning in ChemistryJournal of Chemical Information and Modeling, 2019
- Random forests: A machine learning methodology to highlight the volatile organic compounds involved in olfactory perceptionFood Quality and Preference, 2018
- Machine Learning in Computer-Aided Synthesis PlanningAccounts of Chemical Research, 2018
- Machine Learning, Quantum Chemistry, and Chemical Space2017
- Prediction of interaction between small molecule and enzyme using AdaBoostMolecular Diversity, 2009