ULViT-GAN: Advancing Stain Transfer in H&E and IHC Pathology Images with a UNet-like Vision-Transformer GAN

Authors:
Open Access
Publisher website
Google Scholar
Abstract
This study addresses the challenge of limited access to Her2 testing for breast cancer patients due to high costs. In resource-constrained regions, pathologists heavily rely on cost-effective Hematoxylin and Eosin (H&E) stained images for diagnosis. However, H&E data alone is insufficient for accurate breast cancer typing. Leveraging advancements in deep learning, the paper introduces Unet-like Vision-Transformer GAN (ULViT-GAN), a model integrating Vision Transformer and U-Net with a Harr wavelet module for generating high-pathological-consistency Her2 images from H&E data. Trained on a large dataset, ULViT-GAN outperforms UVCGAN, reducing FID and KID scores by 15.68% and 27.35% (H&E to Her2), and 17.72% and 15.87% (Her2 to H&E). This model offers a cost-effective solution for accurate breast cancer diagnosis, particularly in resource-limited areas, potentially enhancing patient survival rates.