OrderRex: clinical order decision support and outcome predictions by data-mining electronic medical records
Authors:
Open Access
- 21 July 2015
- journal article
- research article
- Published by Oxford University Press (OUP) in Journal of the American Medical Informatics Association
- Vol. 23 (2) , 339-348
Abstract
Objective: To answer a “grand challenge” in clinical decision support, the authors produced a recommender system that automatically data-mines inpatient decision support from electronic medical records (EMR), analogous to Netflix or Amazon.com’s product recommender. Materials and Methods: EMR data were extracted from 1 year of hospitalizations (>18K patients with >5.4M structured items including clinical orders, lab results, and diagnosis codes). Association statistics were counted for the ∼1.5K most common items to drive an order recommender. The authors assessed the recommender’s ability to predict hospital admission orders and outcomes based on initial encounter data from separate validation patients. Results: Compared to a reference benchmark of using the overall most common orders, the recommender using temporal relationships improves precision at 10 recommendations from 33% to 38% ( P < 10 −10 ) for hospital admission orders. Relative risk-based association methods improve inverse frequency weighted recall from 4% to 16% ( P < 10 −16 ). The framework yields a prediction receiver operating characteristic area under curve (c-statistic) of 0.84 for 30 day mortality, 0.84 for 1 week need for ICU life support, 0.80 for 1 week hospital discharge, and 0.68 for 30-day readmission. Discussion: Recommender results quantitatively improve on reference benchmarks and qualitatively appear clinically reasonable. The method assumes that aggregate decision making converges appropriately, but ongoing evaluation is necessary to discern common behaviors from “correct” ones. Conclusions: Collaborative filtering recommender algorithms generate clinical decision support that is predictive of real practice patterns and clinical outcomes. Incorporating temporal relationships improves accuracy. Different evaluation metrics satisfy different goals (predicting likely events vs. “interesting” suggestions).Keywords
This publication has 46 references indexed in Scilit:
- Practice-Based Evidence: Profiling the Safety of Cilostazol by Text-Mining of Clinical NotesPLOS ONE, 2013
- Risk Prediction Models for Hospital ReadmissionJAMA, 2011
- Automatic detection of omissions in medication listsJournal of the American Medical Informatics Association, 2011
- An Automated Model to Identify Heart Failure Patients at Risk for 30-Day Readmission or Death Using Electronic Medical Record DataMedical Care, 2010
- Derivation and validation of an index to predict early death or unplanned readmission after discharge from hospital to the communityCanadian Medical Association Journal, 2010
- Likelihood ratios for genome medicineGenome Medicine, 2010
- Distribution of Problems, Medications and Lab Results in Electronic Health Records: The Pareto Principle at WorkApplied Clinical Informatics, 2010
- Guidelines for the diagnosis and treatment of pulmonary hypertension: The Task Force for the Diagnosis and Treatment of Pulmonary Hypertension of the European Society of Cardiology (ESC) and the European Respiratory Society (ERS), endorsed by the International Society of Heart and Lung Transplantation (ISHLT)European Heart Journal, 2009
- Risk stratification of early admission to the intensive care unit of patients with no major criteria of severe community-acquired pneumonia: development of an international prediction ruleCritical Care, 2009
- Grand challenges in clinical decision supportJournal of Biomedical Informatics, 2008