Integrating Natural Language Processing With ChatGPT to Improve Quality of Artificial Intelligence Summaries of Prostate Cancer Consultations.
- 8 July 2026
- journal article
- Published by American Society of Clinical Oncology (ASCO) in JCO Clinical Cancer Informatics
- Vol. 10 (3) , e2500199
Abstract
Patients with prostate cancer (PC) face complex treatment decisions and often struggle to recall information from consultations. Artificial intelligence(AI)-generated summaries have potential to improve recall, but summaries of key concepts from unstructured transcripts often lack focus and quality. We investigated whether natural language processing (NLP) preprocessing before AI summarization improves topic concordance and quality of risk information in AI summaries. We audio-recorded and transcribed 42 PC consultations. Validated NLP models identified sentences about 5 content areas important for PC decision making: cancer prognosis (CP), life expectancy (LE), erectile dysfunction (ED), irritative urinary symptoms (IUS), and urinary incontinence (UI). ChatGPT-4.0 summarized raw transcripts and NLP-identified sentences across NLP probability thresholds from 50% to 90%. Summaries were reviewed for topic concordance, quality of risk information, and risk quantification. Logistic regression evaluated associations of these outcomes with NLP thresholds. We analyzed 1,260 ChatGPT-4.0 summaries (6 per topic across 42 consultations). For each 10% increase in NLP probability threshold, the proportion of topic-related sentences in ChatGPT summaries increased by 18% for ED (incidence rate ratio [IRR], 1.18, 95% CI, 1.1 to 1.2), 16% for UI (IRR, 1.16, 95% CI, 1.1 to 1.1), 31% for IUS (IRR, 1.31; 95% CI, 1.2 to 1.3), 30% for LE (IRR, 1.30, 95% CI, 1.2 to 1.3), and 12% for CP (IRR, 1.12, 95% CI, 1.1 to 1.1). Odds of risk quantification increased for LE (odds ratio [OR], 1.39, 95% CI, 1.26 to 1.54, P < .001) and UI (OR, 1.18, 95% CI, 1.08 to 1.28, P < .001). Quality of risk information significantly increased for LE (0.16 points, 95% CI, 0.12 to 0.20) for each 10% increase in NLP threshold, with a clinically meaningful change (1 point) at a 60% threshold. NLP preprocessing improves topic concordance, quality of risk information, and quantification in AI summaries of select content areas important for PC decision making, with potential to improve clinical care.Keywords
This publication has 17 references indexed in Scilit:
- Development and validation of a natural language processing system to assess quality of physician communication in prostate cancer consultationsProstate Cancer and Prostatic Diseases, 2025
- Using ChatGPT for writing hospital inpatient discharge summaries – perspectives from an inpatient infectious diseases serviceBMC Health Services Research, 2025
- Using natural language processing to analyse text data in behavioural scienceNature Reviews Psychology, 2025
- Variation in communication of side effects in prostate cancer treatment consultationsProstate Cancer and Prostatic Diseases, 2024
- Variation in Communication of Competing Risks of Mortality in Prostate Cancer Treatment ConsultationsJournal of Urology, 2022
- Cancer information overload: Discriminant validity and relationship to sun safe behaviorsPatient Education and Counseling, 2020
- Radical Prostatectomy or Watchful Waiting in Prostate Cancer — 29-Year Follow-upNew England Journal of Medicine, 2018
- Clinically Localized Prostate Cancer: AUA/ASTRO/SUO Guideline. Part I: Risk Stratification, Shared Decision Making, and Care OptionsJournal of Urology, 2018
- AUA White Paper on Implementation of Shared Decision Making into Urological PracticeUrology Practice, 2016
- The cancer information overload (CIO) scale: Establishing predictive and discriminant validityPatient Education and Counseling, 2014