FG-BERT: a generalized and self-supervised functional group-based molecular representation learning framework for properties prediction
Authors:
- 22 September 2023
- journal article
- research article
- Published by Oxford University Press (OUP) in Briefings in Bioinformatics
- Vol. 24 (6)
Abstract
Artificial intelligence-based molecular property prediction plays a key role in molecular design such as bioactive molecules and functional materials. In this study, we propose a self-supervised pretraining deep learning (DL) framework, called functional group bidirectional encoder representations from transformers (FG-BERT), pertained based on ~1.45 million unlabeled drug-like molecules, to learn meaningful representation of molecules from function groups. The pretrained FG-BERT framework can be fine-tuned to predict molecular properties. Compared to state-of-the-art (SOTA) machine learning and DL methods, we demonstrate the high performance of FG-BERT in evaluating molecular properties in tasks involving physical chemistry, biophysics and physiology across 44 benchmark datasets. In addition, FG-BERT utilizes attention mechanisms to focus on FG features that are critical to the target properties, thereby providing excellent interpretability for downstream training tasks. Collectively, FG-BERT does not require any artificially crafted features as input and has excellent interpretability, providing an out-of-the-box framework for developing SOTA models for a variety of molecule (especially for drug) discovery tasks.Keywords
Funding Information
- Natural Science Foundation of Guangdong Province (2023B1515020042)
- National Natural Science Foundation of China (81973241)
This publication has 43 references indexed in Scilit:
- Choosing Feature Selection and Learning Algorithms in QSARJournal of Chemical Information and Modeling, 2014
- A Bayesian Approach to in Silico Blood-Brain Barrier Penetration ModelingJournal of Chemical Information and Modeling, 2012
- ChEMBL: a large-scale bioactivity database for drug discoveryNucleic Acids Research, 2011
- The Harvard Clean Energy Project: Large-Scale Computational Screening and Design of Organic Photovoltaics on the World Community GridThe Journal of Physical Chemistry Letters, 2011
- Thousands of chemical starting points for antimalarial lead identificationNature, 2010
- Extended-Connectivity FingerprintsJournal of Chemical Information and Modeling, 2010
- Recent advances in computer-aided drug designBriefings in Bioinformatics, 2009
- Cell biology, regulation and inhibition of β‐secretase (BACE‐1)The FEBS Journal, 2009
- ESOL: Estimating Aqueous Solubility Directly from Molecular StructureJournal of Chemical Information and Computer Sciences, 2004
- Survey of the QSAR and in vitro approaches for developing non-animal methods to supersede the in vivo LD50 testFood and Chemical Toxicology, 1990