WelQrate: Defining the Gold Standard in Small Molecule Drug Discovery Benchmarking
Authors:
Open Access
- 1 January 2024
- conference paper
- Published by Curran Associates, Inc. in Biophysical Journal
- p. 53222-53236
Abstract
While deep learning has revolutionized computer-aided drug discovery, the AI community has predominantly focused on model innovation and placed less emphasis on establishing best benchmarking practices. We posit that without a sound model evaluation framework, the AI community’s efforts cannot reach their full potential, thereby slowing the progress and transfer of innovation into real-world drug discovery. Thus, in this paper, we seek to establish a new gold standard for small molecule drug discovery benchmarking, WelQrate. Specifically, our contributions are threefold: WelQrate Dataset Collection - we introduce a meticulously curated collection of 9 datasets spanning 5 therapeutic target classes. Our hierarchical curation pipelines, designed by drug discovery experts, go beyond the primary high-throughput screen by leveraging additional confirmatory and counter screens along with rigorous domain-driven preprocessing, such as Pan-Assay Interference Compounds (PAINS) filtering, to ensure the high-quality data in the datasets; WelQrate Evaluation Framework - we propose a standardized model evaluation framework considering high-quality datasets, featurization, 3D conformation generation, evaluation metrics, and data splits, which provides a reliable benchmarking for drug discovery experts conducting real-world virtual screening; Benchmarking - we evaluate model performance through various research questions using the WelQrate dataset collection, exploring the effects of different models, dataset quality, featurization methods, and data splitting strategies on the results. In summary, we recommend adopting our proposed WelQrate as the gold standard in small molecule drug discovery benchmarking. The WelQrate dataset collection, along with the curation codes, and experimental scripts are all publicly available at WelQrate.org.Keywords
All Related Versions
This publication has 20 references indexed in Scilit:
- Computational Methods in Drug DiscoveryPharmacological Reviews, 2014
- Hit Identification and Optimization in Virtual Screening: Practical Recommendations Based on a Critical Literature AnalysisJournal of Medicinal Chemistry, 2013
- Benchmarking Ligand-Based Virtual High-Throughput Screening with the PubChem DatabaseMolecules, 2013
- Conformations and 3D pharmacophore searchingDrug Discovery Today: Technologies, 2010
- Trust, But Verify: On the Importance of Chemical Structure Curation in Cheminformatics and QSAR Modeling ResearchJournal of Chemical Information and Modeling, 2010
- New Substructure Filters for Removal of Pan Assay Interference Compounds (PAINS) from Screening Libraries and for Their Exclusion in BioassaysJournal of Medicinal Chemistry, 2010
- Glide: A New Approach for Rapid, Accurate Docking and Scoring. 2. Enrichment Factors in Database ScreeningJournal of Medicinal Chemistry, 2004
- Experimental and computational approaches to estimate solubility and permeability in drug discovery and development settings 1PII of original article: S0169-409X(96)00423-1. The article was originally published in Advanced Drug Delivery Reviews 23 (1997) 3–25. 1Advanced Drug Delivery Reviews, 2001
- Improved Scoring of Ligand−Protein Interactions Using OWFEG Free Energy GridsJournal of Medicinal Chemistry, 2001
- The Properties of Known Drugs. 1. Molecular FrameworksJournal of Medicinal Chemistry, 1996