AI-VOICE: A Method to Measure and Incorporate Patient Utilities Into AI-Informed Healthcare Workflows

Authors:
Preprint
Publisher website
Google Scholar
Abstract
Background: Patients are important participants in their medical care, yet artificial intelligence (AI) models are used to guide care with minimal patient input. This limitation is made partially worse due to a paucity of rigorous methods to measure and incorporate patient values of the tradeoffs inherent in AI applications. This paper presents AI-VOICE (Values-Oriented Implementation and Context Evaluation), a novel method to collect patient values, or utilities, of the downstream consequences stemming from an AI model’s use to guide care. The results are then used to select the model’s risk threshold, offering a mechanism by which an algorithm can concretely reflect patient values. Methods: The entity being evaluated by AI-VOICE is an AI-informed workflow , which is composed of the patient’s health state, an action triggered by the AI model, and the benefits and harms accrued as a consequence of that action. The utilities of these workflows are measured through a survey-based, standard gamble experiment. These utilities define a patient-specific ratio of the cost of an inaccurate prediction versus the benefits of an accurate one. This ratio is mapped to the receiver-operator-characteristic curve to identify the risk threshold that reflects the patient’s values. The survey instrument is made freely available to researchers through a web-based application. Results: A demonstration of AI-VOICE is provided using a hypothetical sepsis prediction algorithm. Conclusion: AI-VOICE offers an accessible, quantitative method to incorporate patient values into AI-informed healthcare workflows.

This publication has 17 references indexed in Scilit: