GRU-SCANET: Unleashing the Power of GRU-based Sinusoidal CApture Network for Precision-driven Named Entity Recognition
Authors:
- Bill GatesBGBill Gates
- Happi HappiHHHappi Happi
- Fokou GeraudFGFokou Geraud
- Danai PelapDPDanai Pelap
- Pierre SymeonidouPSPierre Symeonidou
- Geraud PelapGPGeraud Pelap
- Danai SymeonidouDSDanai Symeonidou
- Pierre LarmandePLPierre Larmande
Preprint
Abstract
Motivation: Pre-trained Language Models (PLMs) have achieved remarkable performance across various natural language processing tasks. However, they encounter challenges in biomedical Named Entity Recognition (NER), such as high computational costs and the need for complex fine-tuning. These limitations hinder the efficient recognition of biological entities, especially within specialized corpora. To address these issues, we introduce GRU-SCANET (Gated Recurrent Unit-based Sinusoidal Capture Network), a novel architecture that directly models the relationship between input tokens and entity classes. Our approach offers a computationally efficient alternative for extracting biological entities by capturing contextual dependencies within biomedical texts. Results: GRU-SCANET combines positional encoding, bidirectional GRUs (BiGRUs), an attention-based encoder, and a conditional random field (CRF) decoder to achieve high precision in entity labeling. This design effectively mitigates the challenges posed by unbalanced data across multiple corpora. Our model consistently outperforms leading benchmarks, achieving better performance than BioBERT (8/8 evaluations), PubMedBERT (5/5 evaluations), and the previous stateof-the-art (SOTA) models (8/8 evaluations), including Bern2 (5/5 evaluations). These results highlight the strength of our approach in capturing token-entity relationships more effectively than existing methods, advancing the state of biomedical NER.Keywords
All Related Versions
This publication has 17 references indexed in Scilit:
- From zero to hero: Harnessing transformers for biomedical named entity recognition in zero- and few-shot contextsArtificial Intelligence in Medicine, 2024
- BERN2: an advanced neural biomedical named entity recognition and normalization toolBioinformatics, 2022
- CNERVis: a visual diagnosis tool for Chinese named entity recognitionJournal of Visualization, 2021
- Niederschlags-Abfluss-Modellierung mit Long Short-Term Memory (LSTM)Österreichische Wasser- und Abfallwirtschaft, 2021
- Crosslingual named entity recognition for clinical de-identification applied to a COVID-19 Italian data setApplied Soft Computing, 2020
- BioBERT: a pre-trained biomedical language representation model for biomedical text miningBioinformatics, 2019
- Using long short-term memory deep neural networks for aspect-based sentiment analysis of Arabic reviewsInternational Journal of Machine Learning and Cybernetics, 2018
- Deep learning with word embeddings improves biomedical named entity recognitionBioinformatics, 2017
- BioCreative V CDR task corpus: a resource for chemical disease relation extractionDatabase: The Journal of Biological Databases and Curation, 2016
- NCBI disease corpus: A resource for disease name recognition and concept normalizationJournal of Biomedical Informatics, 2014