Evidence Inference 2.0: More Data, Better Models
Authors:
Preprint
Abstract
How do we most effectively treat a disease or condition? Ideally, we could consult a database of evidence gleaned from clinical trials to answer such questions. Unfortunately, no such database exists; clinical trial results are instead disseminated primarily via lengthy natural language articles. Perusing all such articles would be prohibitively time-consuming for healthcare practitioners; they instead tend to depend on manually compiled systematic reviews of medical literature to inform care. NLP may speed this process up, and eventually facilitate immediate consult of published evidence. The Evidence Inference dataset was recently released to facilitate research toward this end. This task entails inferring the comparative performance of two treatments, with respect to a given outcome, from a particular article (describing a clinical trial) and identifying supporting evidence. For instance: Does this article report that chemotherapy performed better than surgery for five-year survival rates of operable cancers? In this paper, we collect additional annotations to expand the Evidence Inference dataset by 25\%, provide stronger baseline models, systematically inspect the errors that these make, and probe dataset quality. We also release an abstract only (as opposed to full-texts) version of the task for rapid model prototyping. The updated corpus, documentation, and code for new baselines and evaluations are available at http://evidence-inference.ebm-nlp.com/.Keywords
All Related Versions
This publication has 9 references indexed in Scilit:
- Fact or Fiction: Verifying Scientific Claims2020
- Toward systematic review automation: a practical guide to using machine learning tools in research synthesisSystematic Reviews, 2019
- BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding2019
- ScispaCy: Fast and Robust Models for Biomedical Natural Language Processing2019
- Inferring Which Medical Treatments Work from Reports of Clinical Trials2019
- The impact of patient, intervention, comparison, outcome (PICO) as a search strategy tool on literature search quality: a systematic reviewJournal of the Medical Library Association, 2018
- A Corpus with Multi-Level Annotations of Patients, Interventions and Outcomes to Support Language Processing for Medical Literature2018
- Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation2014
- Seventy-Five Trials and Eleven Systematic Reviews a Day: How Will We Ever Keep Up?PLOS Medicine, 2010