Hiroshi Fujita

Background/Objectives: Vision-language models (VLMs) such as GPT (Generative Pre-Trained Transformer) have shown potential for medical image interpretation; however, challenges remain in generating reliable radiological findings in clinical practice, as exemplified by dental pathologies. This study proposes a Self-correction Loop with Structured Output (SLSO) framework as an integrated processing methodology to enhance the accuracy and reliability of AI-generated findings for jaw cysts in dental panoramic radiographs. Methods: Dental panoramic radiographs with jaw cysts were used to implement a 10-step integrated processing framework incorporating image analysis, structured data generation, tooth number extraction, consistency checking, and iterative regeneration. The framework functioned as an external validation mechanism for GPT outputs. Performance was compared against the conventional Chain-of-Thought (CoT) method across seven evaluation items: transparency, internal structure, borders, root resorption, tooth displacement, relationships with other structures, and tooth number. Results: The SLSO framework improved output accuracy for multiple items compared to the CoT method, with the most notable improvements observed in tooth number identification, tooth displacement detection, and root resorption assessment. In successful cases, consistently structured outputs were achieved after up to five regenerations. The framework enforced explicit negative finding descriptions and suppressed hallucinations, although accurate identification of extensive lesions spanning multiple teeth remained limited. Conclusions: This investigation established the feasibility of the proposed integrated processing methodology and provided a foundation for future validation studies with larger, more diverse datasets.

In lung cytology, screeners and pathologists examine many cells in cytological specimens and describe their corresponding imaging findings. To support this process, our previous study proposed an image-finding generation model based on convolutional neural networks and a transformer architecture. However, further improvements are required to enhance the accuracy of these findings. In this study, we developed a cytology-specific image-finding generation model using a vision transformer and open-source large language models. In the proposed method, a vision transformer pretrained on large-scale image datasets and multiple open-source large language models was introduced and connected through an original projection layer. Experimental validation using 1059 cytological images demonstrated that the proposed model achieved favorable scores on language-based evaluation metrics and good classification performance when cells were classified based on the generated findings. These results indicate that a task-specific model is an effective approach for generating imaging findings in lung cytology.

The objective of this study is to achieve site-specific three-dimensional (3D) automatic segmentation of skeletal muscles in body CT images. We aimed to improve recognition accuracy of nine muscle regions: sternocleidomastoid, erector spinae, trapezius, supraspinatus, rectus abdominis, obliques, quadratus lumborum, psoas major, and iliacus. Then, we focused on utilizing all skeletal muscle areas outside the target recognition regions that were not previously used. Our method trains the 2D U-Net to learn both the target site-specific skeletal muscle region and all other skeletal muscles together. We utilized 30 cases of unenhanced body CT images and performed three-fold cross-validation. The proposed method achieved an average Dice coefficient of 88.37% across nine regions, showing improvements of 25.78% and 1.86% compared to the individual learning of each region (baseline) and the simultaneous learning of erector spinae (previous method), respectively. Comprehensive learning of all skeletal muscle regions successfully improved the accuracy of U-Net-based 3D automatic segmentation for site-specific skeletal muscles in body CT. It leads to enhanced, precise body composition analysis for skeletal muscle regions.

In the diagnosis of lung cancer, imaging findings of lung nodules are essential for benign and malignant classifications. Although numerous studies have investigated the classification of lung nodules, no method has been proposed for obtaining detailed imaging findings. This study aimed to develop a novel method for generating image findings and classifying benign and malignant nodules in chest computed tomography (CT) images using vision–language models. In this study, we collected chest CT images of 77 patients diagnosed with either benign or malignant tumors at Fujita Health University Hospital. For these images, we cropped the regions of interest around the nodules, and a pulmonologist provided the corresponding image findings. We used vision–language models for image captioning to generate image findings. The findings generated by these two models were grammatically correct, with no deviations in notation, as expected from the image findings. Moreover, the descriptions of benign and malignant characteristics were accurately obtained. The bootstrapping language–image pretraining (BLIP) base model achieved an accuracy of 79.2% in classifying nodules, and the bilingual evaluation understudy-4 score for agreement with physician findings was 0.561. These results suggest that the proposed method may be effective for classifying and generating lung nodule findings.

Catheter ablation therapy, which is a treatment for atrial fibrillation (AF), has a higher recurrence rate as AF duration increases. Compared to paroxysmal AF (PAF), sustained AF is known to cause progressive anatomic remodeling of the left atrium, resulting in enlargement and shape changes. In this study, we used contrast-enhanced computed tomography (CT) to classify atrial fibrillation (AF) into paroxysmal atrial fibrillation (PAF) and long-term persistent atrial fibrillation (LSAF), which have particularly different recurrence rates after catheter ablation. Contrast-enhanced CT images of 30 patients with PAF and 30 patients with LSAF were input into six pretrained convolutional neural networks (CNNs) for the binary classification of PAF and LSAF. In this study, we propose a method that can recognize information regarding the body axis direction of the left atrium by inputting five slices near the left atrium. The classification was visualized by obtaining a saliency map based on score-class activation mapping (CAM). Furthermore, we surveyed cardiologists regarding the classification of AF types, and the results of the CNN classification were compared with the results of physicians’ clinical judgment. The proposed method achieved the highest correct classification rate (81.7%). In particular, models with shallow layers, such as VGGNet and ResNet, are able to capture the overall characteristics of the image and therefore are likely to be suitable for focusing on the left atrium. In many cases, patients with an enlarged left atrium tended to have long-lasting AF, confirming the validity of the proposed method. The results of the saliency map and survey of physicians’ basis for judgment showed that many patients tended to focus on the shape of the left atrium in both classifications, suggesting that this method can classify atrial fibrillation more accurately than physicians, similar to the judgment criteria of physicians.

Cytology, a type of pathological examination, involves sampling cells from the human body and observing the morphology of the nucleus, cytoplasm, and cell arrangement. In developing classification AI technologies to support cytology, it is essential to collect and utilize a diverse range of images without bias. However, this is often challenging in practice because of the epidemiologic bias of cancer types and cellular characteristics. The main aim of this study was to develop a method to generate cytological diagnostic images from image findings using text-to-image technology in order to generate diverse images. In the proposed method, we collected Papanicolaou-stained specimens derived from the lung cells of 135 lung cancer patients, from which we extracted 472 patch images. Descriptions of the corresponding findings for these patch images were compiled to create a data set. This dataset was then utilized to finetune the Stable Diffusion (SD) v1 and v2 models. The cell images generated by this method closely resemble real images, and both cytotechnologists and cytopathologists provided positive subjective evaluations. Furthermore, SDv2 demonstrated shapes and contours of nuclei and cytoplasm that were more similar to real images compared to SDv1, showing superior performance in quantitative evaluation metrics. When the generated images were utilized in the classification tasks for cytological images, there was an improvement in classification performance. These results indicate that the proposed method may be effective for generating high-quality cytological images, which enables the image classification model to learn diverse features, thereby improving classification performance.

Deep-learning-based methods can improve robustness against individual variations in computed tomography (CT) images of the sternocleidomastoid muscle, which is a challenge when using conventional methods based on probabilistic atlases are used for automatic segmentation. Thus, this study proposes a novel multiclass learning approach for the joint segmentation of the sternocleidomastoid and skeletal muscles in CT images, and it employs a two-dimensional U-Net architecture. The proposed method concurrently learns and segmented segments the sternocleidomastoid muscle and the entire skeletal musculature. Consequently, three-dimensional segmentation results are generated for both muscle groups. Experiments conducted on a dataset of 30 body CT images demonstrated segmentation accuracies of 82.94% and 92.73% for the sternocleidomastoid muscle and entire skeletal muscle compartment, respectively. These results outperformed those of conventional methods, such as the single-region learning of a target muscle and multiclass learning of specific muscle pairs. Moreover, the multiclass learning paradigm facilitated a robust segmentation performance regardless of the input image range. This highlights the method’s potential for cases that present muscle atrophy or reduced muscle strength. The proposed method exhibits promising capabilities for the high-accuracy joint segmentation of the sternocleidomastoid and skeletal muscles and is effective in recognizing skeletal muscles, thus, it holds promise for integration into computer-aided diagnostic systems for comprehensive musculoskeletal analysis. These findings are expected to enhance medical image analysis techniques and their applications in clinical decision support systems.

Skeletal muscle segmentation of the L3 slice can be used to estimate total body skeletal muscle mass. However, site-specific three-dimensional (3D) segmentation of each region, such as the erector spinae, quadratus lumborum, psoas major, oblique muscle, and rectus abdominis, in the L3 slice has not yet been achieved for the accurate measurement of skeletal muscle mass. Herein, we propose a method for site-specific 3D segmentation of skeletal muscles in the L3 slice from body CT images. We focused on the characteristics of the erector spinae muscle (ESM), which can be simultaneously observed with other skeletal muscles on craniocaudal slices and can be accurately segmented using machine learning. We introduce a segmentation method with ESM (w/ESM) using 2D U-Net for the simultaneous learning of the erector spinae and skeletal muscles, which are recognized as targets. In a three-fold cross-validation using 30 CT image cases, the mean Dice value of the baseline method, without ESM (wo/ESM), was 0.637, whereas that for the segmentation of skeletal muscles by w/ESM was 0.864, an improvement of 0.227. Our method improves the accuracy of site-specific segmentation of skeletal muscles within the L3 slice and helps evaluate the skeletal muscles through 3D imaging. This effect of w/ESM was confirmed for the skeletal muscles within the L3 slice as well as the trapezius and supraspinatus muscles. These results demonstrate the effectiveness of simultaneously learning the erector spinae and skeletal muscles in improving the accuracy of site-specific segmentation of skeletal muscles.

PET/CT can scan low-dose computed tomography (LDCT) images with morphological information and PET images with functional information. Because the whole body is targeted for imaging, PET/CT examinations are important in cancer diagnosis. However, the several images obtained by PET/CT place a heavy burden on radiologists during diagnosis. Thus, the development of computer-aided diagnosis (CAD) and technologies assisting in diagnosis has been requested. However, because FDG accumulation in PET images differs for each organ, recognizing organ regions is essential for developing lesion detection and analysis algorithms for PET/CT images. Therefore, we developed a method for automatically extracting organ regions from PET/CT images using U-Net or DenseUNet, which are deep-learning-based segmentation networks. The proposed method is a hybrid approach combining morphological and functional information obtained from LDCT and PET images. Moreover, pre-training using ImageNet and RadImageNet was performed and compared. The best extraction accuracy was obtained by pre-training ImageNet with Dice indices of 94.1, 93.9, 91.3, and 75.1% for the liver, kidney, spleen, and pancreas, respectively. This method obtained better extraction accuracy for low-quality PET/CT images than did existing studies on PET/CT images and was comparable to existing studies on diagnostic contrast-enhanced CT images using the hybrid method and pre-training.