Follow-up Question Generation For Enhanced Patient-Provider Conversations
Preprint
Abstract
Follow-up question generation is an essential feature of dialogue systems as it can reduce conversational ambiguity and enhance modeling complex interactions. Conversational contexts often pose core NLP challenges such as (i) extracting relevant information buried in fragmented data sources, and (ii) modeling parallel thought processes. These two challenges occur frequently in medical dialogue as a doctor asks questions based not only on patient utterances but also their prior EHR data and current diagnostic hypotheses. Asking medical questions in asynchronous conversations compounds these issues as doctors can only rely on static EHR information to motivate follow-up questions. To address these challenges, we introduce FollowupQ, a novel framework for enhancing asynchronous medical conversation. FollowupQ is a multi-agent framework that processes patient messages and EHR data to generate personalized follow-up questions, clarifying patient-reported medical conditions. FollowupQ reduces requisite provider follow-up communications by 34%. It also improves performance by 17% and 5% on real and synthetic data, respectively. We also release the first public dataset of asynchronous medical messages with linked EHR data alongside 2,300 follow-up questions written by clinical experts for the wider NLP research community.Keywords
All Related Versions
This publication has 22 references indexed in Scilit:
- A Survey on Recent Advances in LLM-Based Multi-turn Dialogue SystemsACM Computing Surveys, 2025
- An Empirical Study of Catastrophic Forgetting in Large Language Models During Continual Fine-TuningIEEE Transactions on Audio, Speech and Language Processing, 2025
- HealthQ: Unveiling questioning capabilities of LLM chains in healthcare conversationsSmart Health, 2025
- Retrieval-Augmented Generation with Knowledge Graphs for Customer Service Question Answering2024
- Using large language model to guide patients to create efficient and comprehensive clinical care messageJournal of the American Medical Informatics Association, 2024
- Understanding diagnostic conversations in clinical practice: A systematic reviewPatient Education and Counseling, 2023
- Death by Patient PortalJAMA, 2023
- Chatbots in customer service: Their relevance and impact on service qualityProcedia Computer Science, 2022
- The State of Telehealth Before and After the COVID-19 PandemicPrimary Care: Clinics in Office Practice, 2022
- Systematic review of the Hawthorne effect: New concepts are needed to study research participation effectsJournal of Clinical Epidemiology, 2014