Systematic benchmarking of 13 AI methods for predicting cyclic peptide membrane permeability
Open Access
- 28 August 2025
- journal article
- research article
- Published by Springer Nature in Journal of Cheminformatics
- Vol. 17 (1) , 1-12
Abstract
Cyclic peptides are promising drug candidates due to their ability to modulate intracellular protein–protein interactions, a property often inaccessible to small molecules. However, their typically poor membrane permeability limits therapeutic applicability. Accurate computational prediction of permeability can accelerate the identification of cell-permeable candidates, reducing reliance on time-consuming and costly experimental screening. Although deep learning has shown potential in predicting molecular properties, its application in permeability prediction remains underexplored. A systematic evaluation of these models is important to assess current capabilities and guide future development. In this study, we conduct a comprehensive benchmark of 13 machine learning models for predicting cyclic peptide membrane permeability. These models cover four types of molecular representations: fingerprints, SMILES strings, molecular graphs, and 2D images. We use experimentally measured PAMPA permeability data from the CycPeptMPDB database, comprising nearly 6000 cyclic peptides, and evaluate performance across three prediction tasks: regression, binary classification, and soft-label classification. Two data-splitting strategies, random split and scaffold split, are used to assess the generalizability of trained models. Our results show that model performance depends strongly on molecular representation and model architecture. Graph-based models, particularly the Directed Message Passing Neural Network (DMPNN), consistently achieve top performance across tasks. Regression generally outperforms classification. Scaffold-based splitting, although intended to more rigorously assess generalization, yields substantially lower model generalizability compared to random splitting. Comparing prediction errors with experimental variability highlights the practical value of current models while also indicating room for further improvement. This study presents the first comprehensive benchmark of 13 machine learning models for predicting the membrane permeability of cyclic peptides. By evaluating multiple molecular representations and task formulations on a curated experimental dataset, the work reveals key performance trends, highlighting the superiority of graph-based models, the advantages of regression over classification, and the limitations of scaffold-based data splitting. These findings offer actionable insights for future model development and evaluation in cyclic peptide research.Keywords
Funding Information
- A*STAR Central Research Fund, Singapore
- A*STAR-WuXi AppTec Solitaire Grant
This publication has 44 references indexed in Scilit:
- CycPeptMPDB: A Comprehensive Database of Membrane Permeability of Cyclic PeptidesJournal of Chemical Information and Modeling, 2023
- Accurate de novo design of membrane-traversing macrocyclesCell, 2022
- Large-Scale Membrane Permeability Prediction of Cyclic Peptides Crossing a Lipid Bilayer Based on Enhanced Sampling Molecular Dynamics SimulationsJournal of Chemical Information and Modeling, 2021
- Cyclic Peptides as Drugs for Intracellular Targets: The Next Frontier in Peptide Therapeutic DevelopmentChemistry – A European Journal, 2020
- Analyzing Learned Molecular Representations for Property PredictionJournal of Chemical Information and Modeling, 2019
- Understanding Cell Penetration of Cyclic PeptidesChemical Reviews, 2019
- Physicochemical QSAR Analysis of Passive Permeability Across Caco-2 MonolayersJournal of Pharmaceutical Sciences, 2019
- Development of a new permeability assay using low‐efflux MDCKII cellsJournal of Pharmaceutical Sciences, 2011
- Improving Oral Bioavailability of Peptides by Multiple N‐Methylation: Somatostatin AnaloguesAngewandte Chemie International Edition, 2008
- Characterization of the human colon carcinoma cell line (Caco-2) as a model system for intestinal epithelial permeabilityGastroenterology, 1989