Managing the effort-value tradeoff for integrating human judgment into surgery duration predictions
Open Access
- 20 August 2026
- journal article
- research article
- Published by Springer Nature in OR Spectrum
- p. 1-39
Abstract
For surgery duration prediction, it has been shown that integrating human judgment into algorithmic prediction models results in a higher accuracy than machine learning based approaches can achieve alone. However, since humans are a scarce resource in healthcare, their involvement in surgery duration prediction has high opportunity costs as their time could be spent more effectively on patient care and other medical tasks. This raises the question of how these costs should influence human involvement in tasks where algorithms perform reasonably well at minimal cost. This paper introduces a framework for selectively integrating human judgment based on the costs of human involvement and prediction uncertainty, which is captured using algorithmic meta features. Our framework identifies instances in surgery duration prediction where human judgment should be integrated in algorithmic predictions and those where it should not. We evaluate our framework using a data set containing 70,610 surgeries from a large university hospital. We observe that while always integrating human judgment leads to an overall higher accuracy than never integrating human judgment, employing the framework can lead to an even higher accuracy with lower human workload. In our case study planning costs could be reduced by 27% compared to the current status quo. Our results indicate that a cost-sensitive allocation policy that excludes human judgment for task instances that are easy for the algorithm allows humans to focus on medical tasks.Keywords
Funding Information
- Universität zu Köln
This publication has 87 references indexed in Scilit:
- Demand planning for the digital supply chain: How to integrate human judgment and predictive analyticsJournal of Operations Management, 2023
- Development and Validation of a Machine Learning Model to Aid Discharge Processes for Inpatient Surgical CareJAMA Network Open, 2019
- Expert-Level Diagnosis of Nonpigmented Skin Cancer by Combined Convolutional Neural NetworksJAMA Dermatology, 2019
- What This Computer Needs Is a PhysicianJAMA, 2018
- Translating Artificial Intelligence Into Clinical CareJAMA, 2016
- Effect of Availability Bias and Reflective Reasoning on Diagnostic Accuracy Among Internal Medicine ResidentsJAMA, 2010
- The Elements of Statistical Learning2009
- The Design and Analysis of Computer Experiments2003
- Hospital Nurse Staffing and Patient Mortality, Nurse Burnout, and Job DissatisfactionJAMA, 2002
- The need for contextual and technical knowledge in judgmental forecastingJournal of Behavioral Decision Making, 1992