Retrieval-Augmented Generation-Enabled Multimodal Large Language Model for Histopathologic Grading of Cutaneous Squamous Cell Carcinoma and Melanocytic Nevi

Authors:
Open Access
Publisher website
Google Scholar
Abstract
Background: Histopathologic grading is an essential step in guiding treatment recommendations and follow-up for cutaneous lesions. However, inter-observer variability in determining differentiation of cutaneous squamous cell carcinoma (cSCC) and dysplasia of melanocytic nevi remain a challenge. Retrieval-augmented generation (RAG)-assisted artificial intelligence may offer educational and diagnostic support in this process. Methods: Two isolated RAG pathways using Claude 4.5 Opus, each grounded via ChromaDB vector retrieval of task-specific literature, graded 60 cSCC cases by differentiation and 67 melanocytic nevus cases by dysplasia. Each case was analyzed three times, with the majority result used as the final grade. Concordance with reference dermatopathologist grading was calculated with 95% Wilson score confidence intervals; inter-rater reliability was assessed using Cohen’s kappa. Results: For cSCC, the LLM achieved 90.0% concordance (95% CI, 79.9–95.3%) with excellent agreement (κ = 0.85, 95% CI, 0.74–0.96). For nevi, concordance was 44.8% (95% CI, 33.5–56.6%) with agreement not statistically distinguishable from chance (κ = 0.072, 95% CI, −0.10–0.24). Discordant nevus classifications skewed toward the moderately dysplastic category (65.7% of LLM classifications vs. 41.8% of reference classifications), and no severely dysplastic case was concordantly classified (0/6). Conclusions: Despite an identical RAG architecture, concordance with reference dermatopathologist grading was markedly task-dependent: high for cSCC differentiation, low for nevus dysplasia grading. This pattern parallels reported human inter-rater reliability for these tasks, suggesting that task-specific validation is needed before clinical or educational use.

This publication has 26 references indexed in Scilit: