A panoptic segmentation dataset and deep-learning approach for explainable scoring of tumor-infiltrating lymphocytes
Open Access
- 28 June 2024
- journal article
- research article
- Published by Springer Nature in npj Breast Cancer
- Vol. 10 (1) , 1-10
Abstract
Tumor-Infiltrating Lymphocytes (TILs) have strong prognostic and predictive value in breast cancer, but their visual assessment is subjective. To improve reproducibility, the International Immuno-oncology Working Group recently released recommendations for the computational assessment of TILs that build on visual scoring guidelines. However, existing resources do not adequately address these recommendations due to the lack of annotation datasets that enable joint, panoptic segmentation of tissue regions and cells. Moreover, existing deep-learning methods focus entirely on either tissue segmentation or cell nuclei detection, which complicates the process of TILs assessment by necessitating the use of multiple models and reconciling inconsistent predictions. We introduce PanopTILs, a region and cell-level annotation dataset containing 814,886 nuclei from 151 patients, openly accessible at: sites.google.com/view/panoptils. Using PanopTILs we developed MuTILs, a neural network optimized for assessing TILs in accordance with clinical recommendations. MuTILs is a concept bottleneck model designed to be interpretable and to encourage sensible predictions at multiple resolutions. Using a rigorous internal-external cross-validation procedure, MuTILs achieves an AUROC of 0.93 for lymphocyte detection and a DICE coefficient of 0.81 for tumor-associated stroma segmentation. Our computational score closely matched visual scores from 2 pathologists (Spearman R = 0.58–0.61, p < 0.001). Moreover, computational TILs scores had a higher prognostic value than visual scores, independent of TNM stage and patient age. In conclusion, we introduce a comprehensive open data resource and a modeling approach for detailed mapping of the breast tumor microenvironment.Keywords
Funding Information
- U.S. Department of Health & Human Services | NIH | National Cancer Institute (U01CA220401, U24CA19436201)
- U.S. Department of Health & Human Services | NIH | National Cancer Institute
This publication has 29 references indexed in Scilit:
- histolab: A Python library for reproducible Digital Pathology preprocessing with automated testingSoftwareX, 2022
- HookNet: Multi-resolution convolutional neural networks for semantic segmentation in histopathology whole-slide imagesMedical Image Analysis, 2021
- Computational pathology definitions, best practices, and recommendations for regulatory guidance: a white paper from the Digital Pathology AssociationThe Journal of Pathology, 2019
- An Integrated TCGA Pan-Cancer Clinical Data Resource to Drive High-Quality Survival Outcome AnalyticsCell, 2018
- The Practice of Surgical Pathology2018
- Prediction models need appropriate internal, internal–external, and external validationJournal of Clinical Epidemiology, 2016
- Clinical relevance of host immunity in breast cancer: from TILs to the clinicNature Reviews Clinical Oncology, 2015
- U-Net: Convolutional Networks for Biomedical Image Segmentation2015
- Comprehensive molecular portraits of human breast tumoursNature, 2012
- The American Cancer Society Cancer Prevention Study II Nutrition CohortCancer, 2002