Identifying Biomedical Entities for Datasets in Scientific Articles: 4-Step Cache-Augmented Generation Approach Using GPT-4o and PubTator 3.0
Open Access
- 20 November 2025
- journal article
- Published by JMIR Publications Inc. in JMIR Formative Research
- Vol. 9 (1) , e73822
Abstract
Background: The accurate extraction of biomedical entities in scientific articles is essential for effective metadata annotation of research datasets, ensuring data findability, accessibility, interoperability, and reusability in collaborative research. Objective: This study aimed to introduce a novel 4-step cache-augmented generation approach to identify biomedical entities for an automated metadata annotation of datasets, leveraging GPT-4o and PubTator 3.0. Methods: The method integrates four steps: (1) generation of candidate entities using GPT-4o, (2) validation via PubTator 3.0, (3) term extraction based on a metadata schema developed for the specific research area, and (4) a combined evaluation of PubTator-validated and schema-related terms. Applied to 23 articles published in the context of the Collaborative Research Center OncoEscape , the process was validated through supervised, face-to-face interviews with article authors, allowing an assessment of annotation precision using random-effects meta-analysis. Results: The approach yielded a mean of schema-related and PubTator-validated biomedical entities per article. Within the study’s specific context, the overall annotation precision was 98% (95% CI 94%-100%), with most prediction errors concentrated in articles outside the primary basic research domain of the schema. In a subsample (n=20), available supplemental material was included in the prediction process, but it did not improve precision (98%, 95% CI 95%‐100%). Moreover, the mean number of schema-related entities was ( P =.56) and the mean number of PubTator-validated entities was ( P =.68); these values did not increase with the additional information provided in the supplement. Conclusions: This study highlights the potential of large language model–supported metadata annotation. The findings underscore the practical feasibility of full-text analysis and suggest its potential for integration into routine workflows for biomedical metadata generation.Keywords
This publication has 50 references indexed in Scilit:
- Acute suppression of mitochondrial ATP production prevents apoptosis and provides an essential signal for NLRP3 inflammasome activationImmunity, 2024
- Prostaglandin E2 controls the metabolic adaptation of T cells to the intestinal microenvironmentNature Communications, 2024
- ROCK1/2 signaling contributes to corticosteroid-refractory acute graft-versus-host diseaseNature Communications, 2024
- In Vivo Metabolic Imaging of [1‐13C]Pyruvate‐d3 Hyperpolarized By Reversible Exchange With Parahydrogen**Angewandte Chemie International Edition, 2023
- Functional characterization of the PI3K/AKT/MTOR signaling pathway for targeted therapy in B-precursor acute lymphoblastic leukemiaCancer Gene Therapy, 2022
- Therapy response of glucocorticoid-refractory acute GVHD of the lower intestinal tractBone Marrow Transplantation, 2022
- Efficacy and safety of itacitinib versus placebo in combination with corticosteroids for initial treatment of acute graft-versus-host disease (GRAVITAS-301): a randomised, multicentre, double-blind, phase 3 trialThe Lancet Haematology, 2022
- Demethylating therapy increases anti-CD123 CAR T cell cytotoxicity against acute myeloid leukemiaNature Communications, 2021
- Meta-Analysis of Proportions2021
- Data-sharing: Everything on displayNature, 2013