Eunsu Kim

Vision language models (VLM) have achieved success in both natural language comprehension and image recognition tasks. However, their use in pathology report generation for whole slide images (WSIs) is still limited due to the huge size of multi-scale WSIs and the high cost of WSI annotation. Moreover, in most of the existing research on pathology report generation, sufficient validation regarding clinical efficacy has not been conducted. Herein, we propose a novel Patient-level Multi-organ Pathology Report Generation (PMPRG) model, which utilizes the multi-scale WSI features from our proposed multi-scale regional vision transformer (MR-ViT) model and their real pathology reports to guide VLM training for accurate pathology report generation. The model then automatically generates a report based on the provided key features attended regional features. We assessed our model using a WSI dataset consisting of multiple organs, including the colon and kidney. Our model achieved a METEOR score of 0.68, demonstrating the effectiveness of our approach. This model allows pathologists to efficiently generate pathology reports for patients, regardless of the number of WSIs involved.

Perineural invasion (PNI) is a well-established independent prognostic factor for poor outcomes in colorectal cancer (CRC). However, PNI detection in CRC is a cumbersome and time-consuming process, with low inter-and intra-rater agreement. In this study, a deep-learning-based approach was proposed for detecting PNI using histopathological images. We collected 530 regions of histology from 77 whole-slide images (PNI, 100 regions; non-PNI, 430 regions) for training. The proposed hybrid model consists of two components: a segmentation network for tumor and nerve tissues, and a PNI classifier. Unlike a “black-box” model that is unable to account for errors, the proposed approach enables false predictions to be explained and addressed. We presented a high performance, automated PNI detector, with the area under the curve (AUC) for the receiver operating characteristic (ROC) curve of 0.92. Thus, the potential for the use of deep neural networks in PNI screening was proved, and a possible alternative to conventional methods for the pathologic diagnosis of CRC was provided.