RadVLM: a multitask conversational vision-language model for radiology
Top Cited Papers
Open Access
- 25 August 2026
- journal article
- research article
- Published by Springer Nature in Scientific Reports
- Vol. 16 (1)
Abstract
The widespread use of chest X-rays (CXRs), coupled with a shortage of radiologists, has driven growing interest in automated CXR analysis and AI-assisted reporting. While existing vision-language models (VLMs) show promise in specific tasks such as report generation or abnormality detection, they often lack support for interactive diagnostic capabilities. In this work we present RadVLM, a compact, multitask conversational VLM for CXR interpretation. We construct and standardize a large-scale CXR instruction dataset comprising over 1 million image-instruction pairs from multiple public datasets. The dataset integrates single-turn tasks—including report generation, abnormality classification, and visual grounding—with synthetic multi-turn conversations generated from structured CXR attributes. After fine-tuning RadVLM on this instruction dataset, we evaluate it across different tasks together with re-implemented baseline VLMs. Among the evaluated baselines, RadVLM achieves the strongest performance in conversational capabilities and visual grounding, while remaining competitive in other radiology tasks. Ablations comparing task-specific fine-tuning with full multitask fine-tuning are consistent with a benefit of joint training, particularly for lower-resource grounding and conversational settings. Together, these findings support RadVLM as a research prototype for structured CXR interpretation and conversational capabilities to support more effective and accessible diagnostic workflows.Keywords
All Related Versions
Funding Information
- University of Zurich
This publication has 40 references indexed in Scilit:
- SP-Det: Self-prompted dual-text fusion for generalized multi-label lesion detectionKnowledge-Based Systems, 2026
- ChEX: Interactive Localization and Region Description in Chest X-Rays2024
- DualAttNet: Synergistic fusion of image-level and fine-grained disease attention for multi-label lesion detection in chest X-raysComputers in Biology and Medicine, 2023
- Radiologist burnout: Trends in medical imaging utilization under the national health insurance system with the universal code bundling strategy in an academic tertiary medical centreEuropean Journal of Radiology, 2022
- Making the Most of Text Semantics to Improve Biomedical Vision–Language Processing2022
- Any-Resolution Training for High-Resolution Image Synthesis2022
- An Artificial Intelligence–Based Chest X-ray Model on Human Nodule Detection Accuracy From a Multicenter StudyJAMA Network Open, 2021
- Weighted boxes fusion: Ensembling boxes from different object detection modelsImage and Vision Computing, 2021
- Automated radiology report generation using conditioned transformersInformatics in Medicine Unlocked, 2021
- PadChest: A large chest x-ray image dataset with multi-label annotated reportsMedical Image Analysis, 2020