A Comparative Evaluation of Computational Models for RNA modification detection using Nanopore sequencing with RNA004 Chemistry
Preprint
Abstract
Direct RNA sequencing from Oxford Nanopore Technologies (ONT) has become a valuable method for studying RNA modifications such as N6-methyladenosine (m6A), pseudouridine (ψ), and 5-methylcytosine (m5C). Recent advancements in the RNA004 chemistry substantially reduce sequencing errors compared to previous chemistries (e.g., RNA002), thereby promising enhanced accuracy for epitranscriptomic analysis. In this study, we benchmark the performance of two state-of-the-art RNA modification detection models capable of handling RNA004 data - ONT’s Dorado and m6Anet - using two wild-type (WT) cell lines, HEK293T and HeLa, with respective ground truths from GLORI and eTAM-seq, and their paired in vitro transcribed (IVT) RNA as negative controls. We found that under default settings and considering sites with ≥10% modification ratio and ≥10X coverage, Dorado has higher recall (∼0.92) than m6Anet (∼0.51) for m6A detection. Among the overlapping methylated sites between ground truth and computational predictions, there are high correlations of site-specific m6A modification stoichiometry, with correlation coefficient of ∼0.89 for Dorado-truth comparison and ∼0.72 for m6Anet-truth comparison. However, combined assessment of WT and IVT datasets show that while the per-site false positive rate (FPR) can be lower (∼8% for Dorado and ∼33% for m6Anet), both computational tools can have high per-site false discovery rate (FDR) of m6A (∼40% for Dorado and ∼80% for m6Anet) due to the low prevalence of m6A in transcriptome, with a similar trend observed for pseudouridine (∼95% FDR for Dorado). Additional motif analysis reveals that both Dorado and m6Anet exhibit high heterogeneity of false positive calls across sequence contexts, suggesting that sequence contexts help determine accuracy of specific modification calls. There is also a substantial overlap of false positive calls between the two IVT samples, suggesting a post-filtering strategy to improve modification calling by compiling a set of low-confidence sites with a probabilistic model from several IVT samples across diverse cells/tissues. Our analysis highlights key strengths and limitations of the current generation of m6A detection algorithms and offers insights into optimizing thresholds and interpretability. The IVT datasets generated by the RNA004 chemistry provides a publicly available benchmark resource for further development and refinement of computational methods.Keywords
This publication has 39 references indexed in Scilit:
- RNA modifications in cellular metabolism: implications for metabolism-targeted therapy and immunotherapySignal Transduction and Targeted Therapy, 2024
- RNA modifications detection by comparative Nanopore direct RNA sequencingNature Communications, 2021
- Recent advances in the detection of base modifications using the Nanopore sequencerJournal of Human Genetics, 2019
- Accurate detection of m6A RNA modifications in native RNA sequencesNature Communications, 2019
- Highly parallel direct RNA sequencing on an array of nanoporesNature Methods, 2018
- Single-nucleotide-resolution mapping of m6A and m6Am throughout the transcriptomeNature Methods, 2015
- Pseudouridine profiling reveals regulated mRNA pseudouridylation in yeast and human cellsNature, 2014
- Transcriptome-wide mapping of N6-methyladenosine by m6A-seq based on immunocapturing and massively parallel sequencingNature Protocols, 2013
- Comprehensive Analysis of mRNA Methylation Reveals Enrichment in 3′ UTRs and near Stop CodonsCell, 2012
- Topology of the human and mouse m6A RNA methylomes revealed by m6A-seqNature, 2012