Abstract
Graph-based approaches are successful for histology image classification tasks but still face many challenges, such as: 1) the lack of nuclei-level labels and the significant variations between histology images make it extremely difficult to extract discriminative high-level nuclei features like nuclei type, texture and micro-environment; 2) graph-based approaches cannot handle large-scale cell graph nodes typically contained in histology images; and 3) graph neural networks (GNNs) struggle to learn the long-range dependency of cell graphs. To address the above challenges, we propose a scalable graph-based framework for multi-organ histology image classification. We develop a two-step MNP supervised training approach to extract discriminative high-level nuclei features for histology images without nuclei-level labels. Additionally, we introduce a nuclei sampling strategy to make our graph-based framework scalable for large-scale cell graphs. Furthermore, we propose HierArchical Transformer Graph Neural Network (HAT-Net+) for cell graph classifications. HAT-Net+ adopts Transformer to model the long-range dependency of cell graphs and a parameter-free approach to adaptively fuse different hierarchical graph representations of each layer. We achieved the state-of-the-art results on four public histology image classification datasets: CRC dataset (100%), Extended CRC dataset (98%), UZH dataset (96.9%) and BACH dataset (88%). Unlike other methods, our approach can be used in various histology image classification tasks, even for images without nuclei-level labels, indicating its potential in cancer diagnosis.
Funding Information
  • Beijing Natural Science Foundation-Haidian Original Innovation (L182034)
  • National Natural Science Foundation of China (61972046, 62002025)
  • Beijing Natural Science Foundation (4202051)
  • Fundamental Research Funds for the Central Universities (2019XD-A12)
  • Scientific Research Seed Fund of Peking University First Hospital (2021SF45)
  • BUPT Excellent Ph.D. Students Foundation (CX2022222, CX2021136)
  • BUPT innovation and entrepreneurship (2022-YC-A138, 2022-YC-A132)

This publication has 31 references indexed in Scilit: