CPT/HCPCS Code Recommendation from Clinical Notes: A Comparative Evaluation of AI Methods
Preprint
Abstract
Automatic coding from clinical notes has been studied extensively for International Classification of Diseases (ICD) codes, yet broad Current Procedural Terminology (CPT) and Healthcare Common Procedure Coding System (HCPCS) recommendation remains comparatively underexplored. Existing studies often focus on one specialty, a limited code vocabulary, or a single model family, leaving it unclear how different artificial intelligence (AI) paradigms perform under a common, clinically meaningful evaluation. We formulate CPT and HCPCS coding as an AI-assisted recommendation task in which a physician or professional coder reviews a short, ranked list of candidate codes supported by the clinical note. Using operative notes from Vanderbilt University Medical Center (VUMC) and discharge summaries from Medical Information Mart for Intensive Care IV (MIMIC-IV), we compare lexical retrieval, Clinical-Longformer, GPT-5.6-Sol, MedGemma-27B, and an inspectable agentic-style retrieve-and-verify system under a controlled review budget. Micro-averaged recall within a fixed number of recommendations measures whether reference codes reach the reviewable list; micro-F1 is reported only where reference labels are sufficiently complete. Zero-shot GPT-5.6-Sol achieves the highest recall within five and ten candidates: 0.717 and 0.800 on VUMC and lower-bound values of 0.689 and 0.738 on MIMIC-IV. The retrieve-and-verify system reaches 0.695 and 0.784 on VUMC and lower-bound values of 0.575 and 0.657 on MIMIC-IV, with a candidate-linked evidence window attached to each retained recommendation. Diagnostic analyses reveal distinct failure sources, including output-length underfilling, confusion among closely related codes, out-of-knowledge-base generation, and incomplete evidence support. These findings establish a systematic evaluation framework for procedure-code recommendation and identify practical requirements for future systems that are accurate, review-efficient, and grounded in clinical evidence.Keywords
Funding Information
- Vanderbilt Innovation Catalyst Fund
This publication has 18 references indexed in Scilit:
- Systematic evaluation of common natural language processing techniques to codify clinical notesPLOS ONE, 2024
- Accurate and Well-Calibrated ICD Code Assignment Through Attention Over Diverse Label Embeddings2024
- Can a Novel Natural Language Processing Model and Artificial Intelligence Automatically Generate Billing Codes From Spine Surgical Operative Notes?Global Spine Journal, 2023
- MIMIC-IV, a freely accessible electronic health record datasetScientific Data, 2023
- MDACE: MIMIC Documents Annotated with Code Evidence2023
- A comparative study of pretrained language models for long clinical textJournal of the American Medical Informatics Association, 2022
- PLM-ICD: Automatic ICD Coding with Pretrained Language Models2022
- Code Synonyms Do Matter: Multiple Synonyms Matching Network for Automatic ICD Coding2022
- Self-Alignment Pretraining for Biomedical Entity Representations2021
- The Probabilistic Relevance Framework: BM25 and BeyondFoundations and Trends® in Information Retrieval, 2009