"Evaluation of artificial intelligence-generated medicine images through comparison of interpretations by students and specialists" (Preprint)
Preprint
- 24 October 2025
- preprint
- Published by JMIR Publications Inc. in JMIR Preprints
Abstract
BACKGROUND Recent breakthroughs in generative artificial intelligence (AI) have enabled the creation of highly realistic images across diverse domains. However, despite this progress, the application and realism of AI-generated medical images remain insufficiently explored. Accurate medical imaging interpretation is critical for diagnosis and treatment, and any ability of AI to produce images indistinguishable from real ones could have significant implications for medical education, research, and clinical practice. OBJECTIVE This study aimed to evaluate the realism of AI-generated medical images from the perspectives of medical professionals and students. Specifically, we investigated whether current generative AI models can produce images that are realistic enough to confuse trained medical experts, and how their performance compares across different models and image types. METHODS We assessed four generative AI models—Gemini, DALL-E 3, Midjourney, and a customized Stable Diffusion model trained on open radiology datasets. Each model generated four axial abdominal CT and four sagittal brain MR images. Four authentic clinical images were added to each set, resulting in 40 images in total. An extended Visual Turing Test survey was conducted with ten participants: five medical students and five specialists in Anesthesiology and Pain Medicine. Participants rated each image on a five-point realism scale, indicating whether they believed it was real or AI-generated. Statistical analyses compared discrimination performance across participant groups, image modalities, and models. RESULTS Medical students struggled to distinguish real from AI-generated abdominal CT images (P = .064), indicating near-chance performance, but significantly differentiated brain MR images (P = .002). In contrast, specialists demonstrated high discrimination accuracy for both abdominal CT and brain MR images (P < .001). Among AI models, Gemini and the custom Stable Diffusion model achieved higher realism scores (mean rank ≈ 5), whereas DALL-E 3 and Midjourney performed poorly (mean rank ≈ 15), reflecting limited suitability of general-purpose models for medical imaging tasks. CONCLUSIONS Current generative AI systems can produce visually convincing medical images, but experienced medical professionals can still reliably detect subtle inconsistencies, indicating that these models have not yet reached the level of realism needed to fully mimic genuine clinical imaging. Moreover, the customized model outperformed general-purpose systems, supporting the feasibility of locally engineered AI solutions for medical institutions. These findings highlight both the potential and current limitations of generative AI in medical imaging, underscoring the importance of domain-specific training for clinical applicability.Keywords
This publication has 16 references indexed in Scilit:
- Beware of diffusion models for synthesizing medical images—a comparison with GANs in terms of memorizing brain MRI and chest x-ray imagesMachine Learning: Science and Technology, 2025
- Enhancing Diagnostic Accuracy in Medical Imaging: Integrating GAN-Based Data Augmentation for Balanced Dataset Creation2024
- TDASD: Generating medically significant fine-grained lung adenocarcinoma nodule CT images based on stable diffusion models with limited sample sizeComputer Methods and Programs in Biomedicine, 2024
- Brain tumor segmentation using synthetic MR images - A comparison of GANs and diffusion modelsScientific Data, 2024
- Exploring ChatGPT and its impact on societyAI and Ethics, 2024
- Utilizing GPT-4 and generative artificial intelligence platforms for surgical education: an experimental study on skin ulcersEuropean Journal of Plastic Surgery, 2024
- GAN and CNN Model for Generation of Synthetic Medical Images and Classification2023
- High-Resolution Image Synthesis with Latent Diffusion Models2022
- How to fool radiologists with generative adversarial networks? A visual turing test for lung cancer diagnosis2018
- GAN-based synthetic brain MR image generation2018