Extracting Drug-drug Interactions from Biomedical Texts using Knowledge Graph Embeddings and Multi-focal Loss
Authors:
Open Access
- 17 October 2022
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 884-893
Abstract
The field of Drug-drug interaction (DDI) aims to detect descriptions of interactions between drugs from biomedical texts. Currently, researchers have extracted DDIs using pre-trained language models such as BERT, which often misclassify two kinds of DDI types, "Effect" and "Int", on the DDIExtraction 2013 corpus because of highly similar expressions. The use of knowledge graphs can alleviate this problem by incorporating different relationships for each, thus allowing them to be distinguished. Thus, we propose a novel framework to integrate the neural network with a knowledge graph, where the features from these components are complementary. Specifically, we take text features at different levels into account in the neural network part. This is done by firstly obtaining a word-level position feature using PubMedBERT together with a convolution neural network, secondly, getting a phrase-level key path feature using a dependency parsing tree, thirdly, using PubMedBERT with an attention mechanism to obtain a sentence-level language feature, and finally, fusing these three kinds of representation into a synthesized feature. We also extract a knowledge feature from a drug knowledge graph which takes just a few minutes to construct, then concatenate the synthesized feature with the knowledge feature, feed the result into a multi-layer perceptron and obtain the result by a softmax classifier. In order to achieve a good integration of the synthesized feature and the knowledge feature, we train the model using a novel multifocal loss function, KGE-MFL, which is based on a knowledge graph embedding. Finally we attain state-of-the-art results on the DDIExtraction 2013 dataset (micro F-score 86.24%) and on the ChemProt dataset (micro F-score 77.75%), which proves our framework to be effective for biomedical relation extraction tasks. In particular, we fill the performance gap (more than 5.57%) between methods that rely on and do not rely on knowledge graph embedding on the DDIExtraction 2013 corpus, when predicting the "Int" type. The implementation code is available at https://github.com/NWU-IPMI/DDIE-KGE-MFL.Keywords
Funding Information
- Program for International Science and Technology Cooperation Projects of Shaanxi Province (2021KW-63)
This publication has 14 references indexed in Scilit:
- BioBERT: a pre-trained biomedical language representation model for biomedical text miningBioinformatics, 2019
- Drug-Drug Interaction Extraction via Recurrent Hybrid Convolutional Neural Networks with an Improved Focal LossEntropy, 2019
- SciBERT: A Pretrained Language Model for Scientific Text2019
- What Does BERT Learn about the Structure of Language?2019
- SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processing2018
- Chemical–gene relation extraction using recursive neural networkDatabase: The Journal of Biological Databases and Curation, 2018
- An attention-based effective neural model for drug-drug interactions extractionBMC Bioinformatics, 2017
- Focal Loss for Dense Object Detection2017
- Extracting Drug-Drug Interactions with Word and Character-Level Recurrent Neural Networks2017
- Dependency-based convolutional neural network for drug-drug interaction extraction2016