Prompt-based Grouping Transformer for Nucleus Detection and Classification
Authors:
Preprint
Abstract
Automatic nuclei detection and classification can produce effective information for disease diagnosis. Most existing methods classify nuclei independently or do not make full use of the semantic similarity between nuclei and their grouping features. In this paper, we propose a novel end-to-end nuclei detection and classification framework based on a grouping transformer-based classifier. The nuclei classifier learns and updates the representations of nuclei groups and categories via hierarchically grouping the nucleus embeddings. Then the cell types are predicted with the pairwise correlations between categorical embeddings and nucleus features. For the efficiency of the fully transformer-based framework, we take the nucleus group embeddings as the input prompts of backbone, which helps harvest grouping guided features by tuning only the prompts instead of the whole backbone. Experimental results show that the proposed method significantly outperforms the existing models on three datasets.Keywords
All Related Versions
This publication has 21 references indexed in Scilit:
- Visual Prompt Tuning2022
- Nucleus classification in histology images using message passing networkMedical Image Analysis, 2022
- DenseRes-Unet: Segmentation of overlapped/clustered nuclei from multi organ histopathology imagesComputers in Biology and Medicine, 2022
- SSMD: Semi-Supervised medical image detection with adaptive consistency and heterogeneous perturbationMedical Image Analysis, 2021
- NAS-SCAM: Neural Architecture Search-Based Spatial and Channel Joint Attention Module for Nuclei Semantic Segmentation and Classification2020
- Hover-Net: Simultaneous segmentation and classification of nuclei in multi-tissue histology imagesMedical Image Analysis, 2019
- RIC-Unet: An Improved Neural Network Based on Unet for Nuclei Segmentation in Histology ImagesIEEE Access, 2019
- The Role of Tumor-Infiltrating Lymphocytes in Development, Progression, and Prognosis of Non–Small Cell Lung CancerJournal of Thoracic Oncology, 2016
- U-Net: Convolutional Networks for Biomedical Image Segmentation2015
- The Hungarian method for the assignment problemNaval Research Logistics Quarterly, 1955