Quantifying Evidence for Competing Biomedical Hypotheses using Large Language Models and Bayesian Analysis
Preprint
Abstract
Science fundamentally depends on the generation and testing of hypotheses, many of them controversial. An explosion in scientific literature has made evaluating hypotheses even within a domain a problem of scale, and risks slowing an already extensive consensus-building process. While this challenge has prompted interest in automated hypothesis evaluation tools, existing methods have not yet proven effective for comparing hypotheses. Here, we introduce KM-GPT-DCH, an algorithm that combines co-occurrence methods with large language models (LLMs) to develop a transparent and reproducible literature-based algorithm to compare controversial hypotheses using a structured scoring approach with Bayesian methods to estimate confidence. When testing the algorithm on historical controversial hypotheses previously decided, KM-GPT-DCH chooses the correct hypothesis with high confidence several years before the scientific community or public do so. We further apply the algorithm to compare twenty unresolved controversial hypothesis pairs providing guidance for future research. The method can help researchers and the public to evaluate biomedical hypotheses such as “Is it more likely that monoamine deficiency or inflammation causes depression?” It can also be used to assess and visualize historical trends in the scientific literature. A web-based implementation of the algorithm is freely available at https://skim.morgridge.org .Keywords
Funding Information
- John W. and Jeanne M. Rowe Center for Research in Virology
- National Institute of Neurological Disorders and Stroke (1R01NS138454)
- National Institute of Neurological Disorders and Stroke (1R21NS140909)
- Eunice Kennedy Shriver National Institute of Child Health and Human Development (P50HD105353)
- Eunice Kennedy Shriver National Institute of Child Health and Human Development (R01HD094715-A1)
This publication has 76 references indexed in Scilit:
- A phenomenology and epistemology of large language models: transparency, trust, and trustworthinessEthics and Information Technology, 2024
- Understanding translational research in schizophrenia: A novel insight into animal modelsMolecular Biology Reports, 2023
- Death of the Hypothesis: Researchers Do Not Report A Priori Beliefs in General Medicine JournalsJournal of General Internal Medicine, 2021
- Association of Genetic and Environmental Factors With Autism in a 5-Country CohortJAMA Psychiatry, 2019
- Effect of Intestinal Microbial Ecology on the Developing BrainJAMA Pediatrics, 2013
- Effects of Conjugated Equine Estrogen in Postmenopausal Women With HysterectomyJAMA, 2004
- Estrogen plus progestin and the risk of coronary heart diseaseACC Current Journal Review, 2003
- Randomized Trial of Estrogen Plus Progestin for Secondary Prevention of Coronary Heart Disease in Postmenopausal WomenJAMA, 1998
- National Institutes of Health Consensus Development Conference Statement on Cervical CancerGynecologic Oncology, 1997
- Helicobacter pylori in Peptic Ulcer DiseaseJAMA, 1994