Beyond the First Read: AI-Assisted Perceptual Error Detection in Chest Radiography Accounting for Interobserver Variability
Preprint
Abstract
Chest radiography is widely used in diagnostic imaging. However, perceptual errors -- especially overlooked but visible abnormalities -- remain common and clinically significant. Current workflows and AI systems provide limited support for detecting such errors after interpretation and often lack meaningful human--AI collaboration. We introduce RADAR (Radiologist--AI Diagnostic Assistance and Review), a post-interpretation companion system. RADAR ingests finalized radiologist annotations and CXR images, then performs regional-level analysis to detect and refer potentially missed abnormal regions. The system supports a "second-look" workflow and offers suggested regions of interest (ROIs) rather than fixed labels to accommodate inter-observer variation. We evaluated RADAR on a simulated perceptual-error dataset derived from de-identified CXR cases, using F1 score and Intersection over Union (IoU) as primary metrics. RADAR achieved a recall of 0.78, precision of 0.44, and an F1 score of 0.56 in detecting missed abnormalities in the simulated perceptual-error dataset. Although precision is moderate, this reduces over-reliance on AI by encouraging radiologist oversight in human--AI collaboration. The median IoU was 0.78, with more than 90% of referrals exceeding 0.5 IoU, indicating accurate regional localization. RADAR effectively complements radiologist judgment, providing valuable post-read support for perceptual-error detection in CXR interpretation. Its flexible ROI suggestions and non-intrusive integration position it as a promising tool in real-world radiology workflows. To facilitate reproducibility and further evaluation, we release a fully open-source web implementation alongside a simulated error dataset. All code, data, demonstration videos, and the application are publicly available at https://github.com/avutukuri01/RADAR.Keywords
All Related Versions
This publication has 25 references indexed in Scilit:
- Human-AI Complementarity in Diagnostic Radiology: The Case of Double ReadingPhilosophy & Technology, 2025
- Deep learning improves physician accuracy in the comprehensive detection of abnormalities on chest X-raysScientific Reports, 2024
- Artificial intelligence-assisted double reading of chest radiographs to detect clinically relevant missed findings: a two-centre evaluationEuropean Radiology, 2024
- Commonly Missed Findings on Chest RadiographsChest, 2022
- Experimental evidence of effective human–AI collaboration in medical decision-makingScientific Reports, 2022
- Next-Generation Artificial Intelligence for DiagnosisJAMA, 2021
- Automated abnormality classification of chest radiographs using deep convolutional neural networksnpj Digital Medicine, 2020
- Fifty years of computer analysis in chest imaging: rule-based, machine learning, deep learningRadiological Physics and Technology, 2017
- Detection of lung cancer on the chest radiograph: a study on observer performanceEuropean Journal of Radiology, 2001
- Interobserver Variability in the Interpretation of Chest Roentgenograms of Patients With Possible PneumoniaArchives of internal medicine (1960), 1994