A neuro-symbolic method for understanding free-text medical evidence
- 6 May 2021
- journal article
- research article
- Published by Oxford University Press (OUP) in Journal of the American Medical Informatics Association
- Vol. 28 (8) , 1703-1711
Abstract
We introduce Medical evidence Dependency (MD)–informed attention, a novel neuro-symbolic model for understanding free-text clinical trial publications with generalizability and interpretability. We trained one head in the multi-head self-attention model to attend to the Medical evidence Ddependency (MD) and to pass linguistic and domain knowledge on to later layers (MD informed). This MD-informed attention model was integrated into BioBERT and tested on 2 public machine reading comprehension benchmarks for clinical trial publications: Evidence Inference 2.0 and PubMedQA. We also curated a small set of recently published articles reporting randomized controlled trials on COVID-19 (coronavirus disease 2019) following the Evidence Inference 2.0 guidelines to evaluate the model’s robustness to unseen data. The integration of MD-informed attention head improves BioBERT substantially in both benchmark tasks—as large as an increase of +30% in the F1 score—and achieves the new state-of-the-art performance on the Evidence Inference 2.0. It achieves 84% and 82% in overall accuracy and F1 score, respectively, on the unseen COVID-19 data. MD-informed attention empowers neural reading comprehension models with interpretability and generalizability via reusable domain knowledge. Its compositionality can benefit any transformer-based architecture for machine reading comprehension of free-text medical evidence.Keywords
Funding Information
- Bridging the semantic gap between research eligibility criteria and clinical data (5R01LM009886-11)
This publication has 16 references indexed in Scilit:
- Evidence appraisal: a scoping review, conceptual framework, and research agendaJournal of the American Medical Informatics Association, 2017
- Automating Biomedical Evidence Synthesis: RobotReviewer2017
- Medical Question Answering for Clinical Decision Support2016
- RobotReviewer: evaluation of a system for automatically assessing bias in clinical trialsJournal of the American Medical Informatics Association, 2015
- Automatic classification of sentences to support Evidence Based MedicineBMC Bioinformatics, 2011
- Beyond Trial Registration: A Global Trial Bank for Clinical Trial ReportingPLOS Medicine, 2005
- Electronic Trial Banks: A Complementary Method for Reporting Randomized TrialsMedical Decision Making, 2000
- Analysis of questions asked by family doctors regarding patient careBMJ, 1999
- Evidence-based medicineSeminars in Perinatology, 1997
- The UMLS Metathesaurus: representing different views of biomedical concepts.1993