A Scalable Graph-Based Framework for Multi-Organ Histology Image Classification
- 16 August 2022
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Journal of Biomedical and Health Informatics
- Vol. 26 (11) , 5506-5517
Abstract
Graph-based approaches are successful for histology image classification tasks but still face many challenges, such as: 1) the lack of nuclei-level labels and the significant variations between histology images make it extremely difficult to extract discriminative high-level nuclei features like nuclei type, texture and micro-environment; 2) graph-based approaches cannot handle large-scale cell graph nodes typically contained in histology images; and 3) graph neural networks (GNNs) struggle to learn the long-range dependency of cell graphs. To address the above challenges, we propose a scalable graph-based framework for multi-organ histology image classification. We develop a two-step MNP supervised training approach to extract discriminative high-level nuclei features for histology images without nuclei-level labels. Additionally, we introduce a nuclei sampling strategy to make our graph-based framework scalable for large-scale cell graphs. Furthermore, we propose HierArchical Transformer Graph Neural Network (HAT-Net+) for cell graph classifications. HAT-Net+ adopts Transformer to model the long-range dependency of cell graphs and a parameter-free approach to adaptively fuse different hierarchical graph representations of each layer. We achieved the state-of-the-art results on four public histology image classification datasets: CRC dataset (100%), Extended CRC dataset (98%), UZH dataset (96.9%) and BACH dataset (88%). Unlike other methods, our approach can be used in various histology image classification tasks, even for images without nuclei-level labels, indicating its potential in cancer diagnosis.Keywords
Funding Information
- Beijing Natural Science Foundation-Haidian Original Innovation (L182034)
- National Natural Science Foundation of China (61972046, 62002025)
- Beijing Natural Science Foundation (4202051)
- Fundamental Research Funds for the Central Universities (2019XD-A12)
- Scientific Research Seed Fund of Peking University First Hospital (2021SF45)
- BUPT Excellent Ph.D. Students Foundation (CX2022222, CX2021136)
- BUPT innovation and entrepreneurship (2022-YC-A138, 2022-YC-A132)
This publication has 31 references indexed in Scilit:
- Xception: Deep Learning with Depthwise Separable Convolutions2017
- A curated collection of tissue microarray images and clinical outcome data of prostate cancer patientsScientific Data, 2017
- A Dataset and a Technique for Generalized Nuclear Segmentation for Computational PathologyIEEE Transactions on Medical Imaging, 2017
- Deep Residual Learning for Image Recognition2016
- You Only Look Once: Unified, Real-Time Object Detection2016
- Patch-Based Convolutional Neural Network for Whole Slide Tissue Image Classification2016
- Going deeper with convolutions2015
- Computational pathology: Challenges and promises for tissue analysisComputerized Medical Imaging and Graphics, 2011
- Statistical Nuclear Texture Analysis in Cancer Research: A Review of Methods and ApplicationsCritical Reviews™ in Oncogenesis, 2008
- The cell graphs of cancerBioinformatics, 2004