TalkToModel: Explaining Machine Learning Models with Interactive Natural Language Conversations
Preprint
Abstract
Machine Learning (ML) models are increasingly used to make critical decisions in real-world applications, yet they have become more complex, making them harder to understand. To this end, researchers have proposed several techniques to explain model predictions. However, practitioners struggle to use these explainability techniques because they often do not know which one to choose and how to interpret the results of the explanations. In this work, we address these challenges by introducing TalkToModel: an interactive dialogue system for explaining machine learning models through conversations. Specifically, TalkToModel comprises of three key components: 1) a natural language interface for engaging in conversations, making ML model explainability highly accessible, 2) a dialogue engine that adapts to any tabular model and dataset, interprets natural language, maps it to appropriate explanations, and generates text responses, and 3) an execution component that constructs the explanations. We carried out extensive quantitative and human subject evaluations of TalkToModel. Overall, we found the conversational system understands user inputs on novel datasets and models with high accuracy, demonstrating the system's capacity to generalize to new situations. In real-world evaluations with humans, 73% of healthcare workers (e.g., doctors and nurses) agreed they would use TalkToModel over baseline point-and-click systems for explainability in a disease prediction task, and 85% of ML professionals agreed TalkToModel was easier to use for computing explanations. Our findings demonstrate that TalkToModel is more effective for model explainability than existing systems, introducing a new category of explainability tools for practitioners. Code & demo released here: https://github.com/dylan-slack/TalkToModel.Keywords
All Related Versions
This publication has 31 references indexed in Scilit:
- Interpretability and fairness evaluation of deep learning models on MIMIC-IV datasetScientific Reports, 2022
- Recent advances and challenges in task-oriented dialog systemsScience China Technological Sciences, 2020
- From local explanations to global understanding with explainable AI for treesNature Machine Intelligence, 2020
- Definitions, methods, and applications in interpretable machine learningProceedings of the National Academy of Sciences, 2019
- Prolific.ac—A subject pool for online experimentsJournal of Behavioral and Experimental Finance, 2018
- Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization2017
- Interpretable Classification Models for Recidivism PredictionJournal of the Royal Statistical Society Series A: Statistics in Society, 2016
- Supersparse linear integer models for optimized medical scoring systemsMachine Learning, 2015
- Developing Dialogue Managers from Limited Amounts of Data2012
- Toward establishing trust in adaptive agents2008