Evaluating the use of non-linear models in data-driven rescoring of peptide-spectrum matches
Preprint
Abstract
In mass spectrometry (MS)-based proteomics, computational tools match acquired tandem MS spectra to peptides from a sequence database. Machine learning increasingly supports this task through peptide-spectrum match (PSM) rescoring, in which a classifier, typically a linear semi-supervised model, refines the initial matching score. However, Mokapot allows the user to choose among different machine learning algorithms of increasing complexity, from the default linear support vector machine (LSVM) to random forest and XGBoost. Here, we use an entrapment approach to assess the effect of this increasing complexity on PSM identification and the accuracy of the estimated false discovery rate (FDR). We show that, while more complex models increase the number of identified PSMs at a fixed FDR threshold, this gain reflects a bias towards random matches from the target proteome database rather than genuine identifications. Indeed, for the most complex model, the entrapment FDR reaches 6.3% instead of the estimated 1% decoy FDR. This bias thus yields overly optimistic FDR estimates, indicating that model complexity in PSM rescoring must be carefully balanced against this overfitting risk.Keywords
Funding Information
- European Union’s Horizon 2020 (956148)
- European Union’s Horizon 2020 (H2020-INFRAIA-2018-1)
- Horizon Europe projects (101080544)
- Horizon Europe projects (101191739)
- Ghent University Concerted Research Action (BOF21/GOA/033)
- CHIST-ERA project (G0GDV23N)
This publication has 33 references indexed in Scilit:
- Rescoring Peptide Spectrum Matches: Boosting Proteomics Performance by Integrating Peptide Property Predictors Into Peptide IdentificationMolecular & Cellular Proteomics, 2024
- Oktoberfest: Open‐source spectral library generation and rescoring pipeline based on PrositProteomics, 2023
- MS2Rescore: Data-Driven Rescoring Dramatically Boosts Immunopeptide Identification RatesMolecular & Cellular Proteomics, 2022
- mokapot: Fast and Flexible Semisupervised Learning for Peptide DetectionJournal of Proteome Research, 2021
- In‐depth characterization of the tomato fruit pericarp proteomeProteomics, 2017
- Fast and Accurate Protein False Discovery Rates on Large-Scale Proteomics Data Sets with Percolator 3.0Journal of the American Society for Mass Spectrometry, 2016
- Ultradeep Human Phosphoproteome Reveals a Distinct Regulatory Nature of Tyr and Ser/Thr-Based SignalingCell Reports, 2014
- Comet: An open‐source MS / MS sequence database search toolProteomics, 2012
- SearchGUI: An open‐source graphical user interface for simultaneous OMSSA and X!Tandem searchesProteomics, 2011
- Empirical Statistical Model To Estimate the Accuracy of Peptide Identifications Made by MS/MS and Database SearchAnalytical Chemistry, 2002