Mark Schuiveling
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Mark Schuiveling has a total of 114 co-authors. Here are the top co-authors by number of shared publications:
- Karlijn P. M. Suijkerbuijk
Department of Medical Oncology, University Medical Center Utrecht, Utrecht University, Heidelberglaan 100, 3584 CG Utrecht, the Netherlands
10 shared publications - Willeke A. M. Blokx
Department of Pathology, University Medical Center Utrecht, Utrecht University, Heidelberglaan 100, 3584 CG Utrecht, the Netherlands
8 shared publications - Mitko Veta
Medical Image Analysis, Department of Biomedical Engineering, Eindhoven University of Technology
7 shared publications - Ellen Kapiteijn
Leiden University Medical Center, Leiden, The Netherlands
6 shared publications - Hans M. Westgeest
Amphia Hospital, Breda, the Netherlands
6 shared publications - Paul Joannes van Diest
Department of Pathology, University Medical Center Utrecht, Utrecht, The Netherlands
6 shared publications
Latest Publications
- Open Access
- 25 May 2026
- Published by Elsevier in European Journal of Cancer
- Vol. 242, 116831
AI-based histopathology analysis predicts checkpoint inhibitor response in advanced melanoma and identifies patterns associated with response
- 25 May 2026
- Published by Elsevier in European Journal of Cancer
- Vol. 242, 116831
Open AccessAI-based histopathology analysis predicts checkpoint inhibitor response in advanced melanoma and identifies patterns associated with response
- Open Access
Associations between spatial distribution of immune cell subsets and clinical outcomes in patients with advanced melanoma treated with immune checkpoint inhibitors: results from the PUMA challenge
Patients with advanced melanoma are treated with immune checkpoint inhibitors (ICIs), yet less than 50% of patients achieve a durable response while all patients are exposed to the risk of severe side effects. Tumor-infiltrating lymphocytes (TILs) in pathology images are associated with ICI outcomes, but manual assessment is subjective. In addition, the predictive value of other immune cell subsets, including plasma cells, neutrophils, histiocytes, and melanophages, remains unclear. We organized the Panoptic segmentation of nUclei and tissue in advanced MelanomA (PUMA) challenge to evaluate whether the spatial localization of TILs and other immune cell subsets on melanoma H&E slides collected before start of treatment was associated with treatment outcomes. Algorithm performance was evaluated on a hidden test set, after which top-ranked algorithms were applied to pre-treatment metastatic whole-slide images from a large, multicenter cohort of patients with advanced melanoma treated with first-line ICIs ( n =1102). Automatically quantified tissue features and immune cell subsets were then associated with clinical outcomes. Top-performing algorithms improved detection of immune cell subsets, although accuracy for rare classes remained limited. Across challenge participants, TIL density showed the most consistent association with treatment response and survival. Associations for stromal TILs were weaker, while plasma cells, histiocytes, melanophages, neutrophils, necrosis and blood vessels did not show independent associations with outcomes. Overall, the results from the PUMA challenge improved the state of the art of immune cell detection in melanoma histopathology and show that intra-tumoral lymphocytes are the immune cell subset most consistently associated with treatment response and survival. Highlights: We organized the first melanoma-specific tissue and nuclei segmentation competition Winning algorithms were applied to 1102 whole-slide images for biomarker analysis Intra-tumoral TILs were associated with response to immune checkpoint inhibitors Other immune cell subsets showed no independent association with treatment outcomes Tissue segmentation on WSIs was limited by low heterogeneity in training data. Graphical
Open AccessAssociations between spatial distribution of immune cell subsets and clinical outcomes in patients with advanced melanoma treated with immune checkpoint inhibitors: results from the PUMA challenge
Patients with advanced melanoma are treated with immune checkpoint inhibitors (ICIs), yet less than 50% of patients achieve a durable response while all patients are exposed to the risk of severe side effects. Tumor-infiltrating lymphocytes (TILs) in pathology images are associated with ICI outcomes, but manual assessment is subjective. In addition, the predictive value of other immune cell subsets, including plasma cells, neutrophils, histiocytes, and melanophages, remains unclear. We organized the Panoptic segmentation of nUclei and tissue in advanced MelanomA (PUMA) challenge to evaluate whether the spatial localization of TILs and other immune cell subsets on melanoma H&E slides collected before start of treatment was associated with treatment outcomes. Algorithm performance was evaluated on a hidden test set, after which top-ranked algorithms were applied to pre-treatment metastatic whole-slide images from a large, multicenter cohort of patients with advanced melanoma treated with first-line ICIs ( n =1102). Automatically quantified tissue features and immune cell subsets were then associated with clinical outcomes. Top-performing algorithms improved detection of immune cell subsets, although accuracy for rare classes remained limited. Across challenge participants, TIL density showed the most consistent association with treatment response and survival. Associations for stromal TILs were weaker, while plasma cells, histiocytes, melanophages, neutrophils, necrosis and blood vessels did not show independent associations with outcomes. Overall, the results from the PUMA challenge improved the state of the art of immune cell detection in melanoma histopathology and show that intra-tumoral lymphocytes are the immune cell subset most consistently associated with treatment response and survival. Highlights: We organized the first melanoma-specific tissue and nuclei segmentation competition Winning algorithms were applied to 1102 whole-slide images for biomarker analysis Intra-tumoral TILs were associated with response to immune checkpoint inhibitors Other immune cell subsets showed no independent association with treatment outcomes Tissue segmentation on WSIs was limited by low heterogeneity in training data. Graphical
- Open Access
- 23 January 2026
- Published by Springer Nature in Communications Medicine
- Vol. 6 (1) , 49
Author Correction: Blood and tissue correlates of steroid non-response in checkpoint inhibition-induced immune-related adverse events
- 23 January 2026
- Published by Springer Nature in Communications Medicine
- Vol. 6 (1) , 49
Open AccessAuthor Correction: Blood and tissue correlates of steroid non-response in checkpoint inhibition-induced immune-related adverse events
- Open Access
- 18 November 2025
- Published by Springer Nature in Communications Medicine
- Vol. 5 (1) , 472
Blood and tissue correlates of steroid non-response in checkpoint inhibition-induced immune-related adverse events
Background High-dose steroids constitute the cornerstone of first-line treatment for immune-related adverse events (irAEs) associated with immune checkpoint inhibitors, but compromise antitumor immunity. A deeper understanding of irAEs and their response to steroids can improve management. Methods Using a multi-omics approach, we investigated blood- and tissue-based correlates of steroid response, focusing on gastro-intestinal irAEs, in the largest cohort to date. Results Here we show clear trends for elevated TC1/TC17 CD8+ T cells and Th1/Th7-associated interleukins before steroid initiation, and persistent (CD8+) T cell activation after initiation of steroids in blood of steroid non-responders. Cross-sectional analysis of colitis tissue suggested lower lymphocyte infiltration within 24 h in steroid responders. Peripheral T cell PD-1 receptor occupancy was unrelated to steroid response. Non-responders’ colitis tissue was enriched with activated CD4+ memory T cells and a pronounced type 1/17 immune response. Conclusions These findings highlight rapid steroid effects on circulating cells and irAE-affected tissue and support that an enhanced type 1/type 17 response may be associated with steroid non-response in irAEs. Validation of these findings in larger cohorts is warranted.
- 18 November 2025
- Published by Springer Nature in Communications Medicine
- Vol. 5 (1) , 472
Open AccessBlood and tissue correlates of steroid non-response in checkpoint inhibition-induced immune-related adverse events
Background High-dose steroids constitute the cornerstone of first-line treatment for immune-related adverse events (irAEs) associated with immune checkpoint inhibitors, but compromise antitumor immunity. A deeper understanding of irAEs and their response to steroids can improve management. Methods Using a multi-omics approach, we investigated blood- and tissue-based correlates of steroid response, focusing on gastro-intestinal irAEs, in the largest cohort to date. Results Here we show clear trends for elevated TC1/TC17 CD8+ T cells and Th1/Th7-associated interleukins before steroid initiation, and persistent (CD8+) T cell activation after initiation of steroids in blood of steroid non-responders. Cross-sectional analysis of colitis tissue suggested lower lymphocyte infiltration within 24 h in steroid responders. Peripheral T cell PD-1 receptor occupancy was unrelated to steroid response. Non-responders’ colitis tissue was enriched with activated CD4+ memory T cells and a pronounced type 1/17 immune response. Conclusions These findings highlight rapid steroid effects on circulating cells and irAE-affected tissue and support that an enhanced type 1/type 17 response may be associated with steroid non-response in irAEs. Validation of these findings in larger cohorts is warranted.
- Open Access
AI-detected tumor-infiltrating lymphocytes for predicting outcomes in anti-PD1 based treated melanoma
Importance: Easy and accessible biomarkers to predict response to immune checkpoint inhibition (ICI)-treated melanoma are limited. Objective: To evaluate artificial intelligence (AI) detected tumor-infiltrating lymphocytes (TILs) on pretreatment melanoma metastases as a biomarker for response and survival in ICI-treated patients. Design: Multicenter cohort study including patients with advanced melanoma treated with first-line anti-PD1 ± anti-CTLA4 between 2016 and 2023. Median follow-up was 36.3 months. Setting: 11 melanoma treatment centers in the Netherlands. Participants: 1,202 patients with advanced cutaneous melanoma. Exposure: All patients received first-line anti-PD1 ± anti-CTLA4. Main Outcome(s) and Measure(s): The percentage of TILs inside manually annotated tumor area in H&E stained pretreatment metastases was determined using the Hover-NeXt model trained and evaluated on an independent melanoma dataset containing 166,718 pathologist-verified manually annotated cells. The primary outcome was objective response rate (ORR); secondary outcomes were progression-free survival (PFS) and overall survival (OS). Correlation with manual TILs, scored according to the guidelines stated by the immune-oncology working group, was evaluated with Spearman correlation coefficients. Logistic regression and Cox proportional regression were conducted, adjusted for age, sex, disease stage, ICI type, BRAF status, brain metastases, LDH level, and performance status. Results: Metastatic melanoma specimens were available for 1,202 patients, of whom 423 received combination therapy. Median TIL percentage was 9.9% (range 0.3% - 69.4%). A 10% increase in TILs was associated with increased ORR (adjusted OR 1.40 [95% 1.23-1.59]), PFS (adjusted HR 0.85 [95% CI 0.79 – 0.92]) and OS (adjusted HR 0.83 [95% CI 0.76 – 0.91]. Results were consistent for both patients treated with anti-PD1 monotherapy and combination treatment with anti-PD1 plus anti-CTLA4. When comparing manual TIL scoring with AI-detected TILs, associations with response and survival were consistently stronger for AI-detected TILs. Conclusions and Relevance: In patients with advanced melanoma, higher levels of AI-detected TILs on pre-treatment H&E slides were independently associated with improved ICI response and survival. Given the accessibility of TIL scoring on routine histology, TILs may serve as a predictive biomarker for ICI outcomes. To facilitate broader validation, the Hover-NeXt architecture and model weights are publicly available. Key points: Question: What is the predictive value of artificial intelligence-detected tumor-infiltrating lymphocytes (TILs) for clinical outcomes in patients with advanced melanoma receiving first-line immune checkpoint inhibition? Findings: In this multicenter cohort of 1202 patients, TILs in pretreatment metastases were quantified using a melanoma-specific publicly available AI model trained on an independent dataset. A 10% increase was associated with response (aOR 1.40 [95% CI 1.23–1.59]), progression-free survival (aHR 0.85 [95% CI 0.79–0.92]), and overall survival (aHR 0.83 [95% CI 0.76–0.91]). Associations were independent of clinical predictors. Meaning: AI-detected TILs in pretreatment melanoma metastases independently correlate with response and survival.
Open AccessAI-detected tumor-infiltrating lymphocytes for predicting outcomes in anti-PD1 based treated melanoma
Importance: Easy and accessible biomarkers to predict response to immune checkpoint inhibition (ICI)-treated melanoma are limited. Objective: To evaluate artificial intelligence (AI) detected tumor-infiltrating lymphocytes (TILs) on pretreatment melanoma metastases as a biomarker for response and survival in ICI-treated patients. Design: Multicenter cohort study including patients with advanced melanoma treated with first-line anti-PD1 ± anti-CTLA4 between 2016 and 2023. Median follow-up was 36.3 months. Setting: 11 melanoma treatment centers in the Netherlands. Participants: 1,202 patients with advanced cutaneous melanoma. Exposure: All patients received first-line anti-PD1 ± anti-CTLA4. Main Outcome(s) and Measure(s): The percentage of TILs inside manually annotated tumor area in H&E stained pretreatment metastases was determined using the Hover-NeXt model trained and evaluated on an independent melanoma dataset containing 166,718 pathologist-verified manually annotated cells. The primary outcome was objective response rate (ORR); secondary outcomes were progression-free survival (PFS) and overall survival (OS). Correlation with manual TILs, scored according to the guidelines stated by the immune-oncology working group, was evaluated with Spearman correlation coefficients. Logistic regression and Cox proportional regression were conducted, adjusted for age, sex, disease stage, ICI type, BRAF status, brain metastases, LDH level, and performance status. Results: Metastatic melanoma specimens were available for 1,202 patients, of whom 423 received combination therapy. Median TIL percentage was 9.9% (range 0.3% - 69.4%). A 10% increase in TILs was associated with increased ORR (adjusted OR 1.40 [95% 1.23-1.59]), PFS (adjusted HR 0.85 [95% CI 0.79 – 0.92]) and OS (adjusted HR 0.83 [95% CI 0.76 – 0.91]. Results were consistent for both patients treated with anti-PD1 monotherapy and combination treatment with anti-PD1 plus anti-CTLA4. When comparing manual TIL scoring with AI-detected TILs, associations with response and survival were consistently stronger for AI-detected TILs. Conclusions and Relevance: In patients with advanced melanoma, higher levels of AI-detected TILs on pre-treatment H&E slides were independently associated with improved ICI response and survival. Given the accessibility of TIL scoring on routine histology, TILs may serve as a predictive biomarker for ICI outcomes. To facilitate broader validation, the Hover-NeXt architecture and model weights are publicly available. Key points: Question: What is the predictive value of artificial intelligence-detected tumor-infiltrating lymphocytes (TILs) for clinical outcomes in patients with advanced melanoma receiving first-line immune checkpoint inhibition? Findings: In this multicenter cohort of 1202 patients, TILs in pretreatment metastases were quantified using a melanoma-specific publicly available AI model trained on an independent dataset. A 10% increase was associated with response (aOR 1.40 [95% CI 1.23–1.59]), progression-free survival (aHR 0.85 [95% CI 0.79–0.92]), and overall survival (aHR 0.83 [95% CI 0.76–0.91]). Associations were independent of clinical predictors. Meaning: AI-detected TILs in pretreatment melanoma metastases independently correlate with response and survival.
- Open Access
- 20 February 2025
- Published by Oxford University Press (OUP) in JNCI: Journal of the National Cancer Institute
- Vol. 117 (6) , 1245-1252
Body composition and checkpoint inhibitor treatment outcomes in advanced melanoma: a multicenter cohort study
Background: The association of body composition with checkpoint inhibitor outcomes in melanoma is a matter of ongoing debate. In this study, we aim to investigate body mass index (BMI) alongside computed tomography (CT)-derived body composition metrics in the largest cohort to date. Methods: Patients treated with first-line anti-PD1 ± anti-CTLA4 for advanced melanoma were retrospectively identified from 11 melanoma centers in The Netherlands. From baseline CT scans, 5 body composition metrics were extracted: subcutaneous adipose tissue index, visceral adipose tissue index, skeletal muscle index, density, and gauge. These metrics were correlated in univariable and multivariable Cox proportional hazards analysis with progression-free survival, overall survival, and melanoma-specific survival (PFS, OS, and MSS). Results: A total of 1471 eligible patients were included. Median PFS and OS were 9.1 and 38.1 months, respectively. Worse PFS was observed in underweight patients (multivariable hazard ratio [HR] = 1.86, 95% CI = 1.14 to 3.06). Furthermore, prolonged OS was observed in patients with higher skeletal muscle density (multivariable HR = 0.88, 95% CI = 0.81 to 0.97) and gauge (multivariable HR = 0.61, 95% CI = 0.82 to 0.998), whereas higher visceral adipose tissue index was associated with worse OS (multivariable HR = 1.12, 95% CI = 1.04 to 1.22). No association with survival outcomes was found for overweight, obesity, or subcutaneous adipose tissue. Conclusion: Our findings suggest that underweight BMI is associated with worse PFS, whereas higher skeletal muscle density and lower visceral adipose tissue index were associated with improved OS. These associations were independent of known prognostic factors, including sex, age, performance status, and extent of disease. No significant association between higher BMI and survival outcomes was observed.
- 20 February 2025
- Published by Oxford University Press (OUP) in JNCI: Journal of the National Cancer Institute
- Vol. 117 (6) , 1245-1252
Open AccessBody composition and checkpoint inhibitor treatment outcomes in advanced melanoma: a multicenter cohort study
Background: The association of body composition with checkpoint inhibitor outcomes in melanoma is a matter of ongoing debate. In this study, we aim to investigate body mass index (BMI) alongside computed tomography (CT)-derived body composition metrics in the largest cohort to date. Methods: Patients treated with first-line anti-PD1 ± anti-CTLA4 for advanced melanoma were retrospectively identified from 11 melanoma centers in The Netherlands. From baseline CT scans, 5 body composition metrics were extracted: subcutaneous adipose tissue index, visceral adipose tissue index, skeletal muscle index, density, and gauge. These metrics were correlated in univariable and multivariable Cox proportional hazards analysis with progression-free survival, overall survival, and melanoma-specific survival (PFS, OS, and MSS). Results: A total of 1471 eligible patients were included. Median PFS and OS were 9.1 and 38.1 months, respectively. Worse PFS was observed in underweight patients (multivariable hazard ratio [HR] = 1.86, 95% CI = 1.14 to 3.06). Furthermore, prolonged OS was observed in patients with higher skeletal muscle density (multivariable HR = 0.88, 95% CI = 0.81 to 0.97) and gauge (multivariable HR = 0.61, 95% CI = 0.82 to 0.998), whereas higher visceral adipose tissue index was associated with worse OS (multivariable HR = 1.12, 95% CI = 1.04 to 1.22). No association with survival outcomes was found for overweight, obesity, or subcutaneous adipose tissue. Conclusion: Our findings suggest that underweight BMI is associated with worse PFS, whereas higher skeletal muscle density and lower visceral adipose tissue index were associated with improved OS. These associations were independent of known prognostic factors, including sex, age, performance status, and extent of disease. No significant association between higher BMI and survival outcomes was observed.
- Top Cited PapersOpen Access
- 6 January 2025
- Published by Oxford University Press (OUP) in GigaScience
- Vol. 14
A novel dataset for nuclei and tissue segmentation in melanoma with baseline nuclei segmentation and tissue segmentation benchmarks
Background: Melanoma is an aggressive form of skin cancer in which tumor-infiltrating lymphocytes (TILs) are a biomarker for recurrence and treatment response. Manual TIL assessment is prone to interobserver variability, and current deep learning models are not publicly accessible or have low performance. Deep learning models, however, have the potential of consistent spatial evaluation of TILs and other immune cell subsets with the potential of improved prognostic and predictive value. To make the development of these models possible, we created the Panoptic Segmentation of nUclei and tissue in advanced MelanomA (PUMA) dataset and assessed the performance of several state-of-the-art deep learning models. In addition, we show how to improve model performance further by using heuristic postprocessing in which nuclei classes are updated based on their tissue localization. Results: The PUMA dataset includes 155 primary and 155 metastatic melanoma hematoxylin and eosin–stained regions of interest with nuclei and tissue annotations from a single melanoma referral institution. The Hover-NeXt model, trained on the PUMA dataset, demonstrated the best performance for lymphocyte detection, approaching human interobserver agreement. In addition, heuristic postprocessing of deep learning models improved the detection of noncommon classes, such as epithelial nuclei. Conclusion: The PUMA dataset is the first melanoma-specific dataset that can be used to develop melanoma-specific nuclei and tissue segmentation models. These models can, in turn, be used for prognostic and predictive biomarker development. Incorporating tissue and nuclei segmentation is a step toward improved deep learning nuclei segmentation performance. To support the development of these models, this dataset is used in the PUMA challenge.
- 6 January 2025
- Published by Oxford University Press (OUP) in GigaScience
- Vol. 14
Top Cited PapersOpen AccessA novel dataset for nuclei and tissue segmentation in melanoma with baseline nuclei segmentation and tissue segmentation benchmarks
Background: Melanoma is an aggressive form of skin cancer in which tumor-infiltrating lymphocytes (TILs) are a biomarker for recurrence and treatment response. Manual TIL assessment is prone to interobserver variability, and current deep learning models are not publicly accessible or have low performance. Deep learning models, however, have the potential of consistent spatial evaluation of TILs and other immune cell subsets with the potential of improved prognostic and predictive value. To make the development of these models possible, we created the Panoptic Segmentation of nUclei and tissue in advanced MelanomA (PUMA) dataset and assessed the performance of several state-of-the-art deep learning models. In addition, we show how to improve model performance further by using heuristic postprocessing in which nuclei classes are updated based on their tissue localization. Results: The PUMA dataset includes 155 primary and 155 metastatic melanoma hematoxylin and eosin–stained regions of interest with nuclei and tissue annotations from a single melanoma referral institution. The Hover-NeXt model, trained on the PUMA dataset, demonstrated the best performance for lymphocyte detection, approaching human interobserver agreement. In addition, heuristic postprocessing of deep learning models improved the detection of noncommon classes, such as epithelial nuclei. Conclusion: The PUMA dataset is the first melanoma-specific dataset that can be used to develop melanoma-specific nuclei and tissue segmentation models. These models can, in turn, be used for prognostic and predictive biomarker development. Incorporating tissue and nuclei segmentation is a step toward improved deep learning nuclei segmentation performance. To support the development of these models, this dataset is used in the PUMA challenge.
- Open Access
Blood and tissue correlates of steroid non-response in checkpoint inhibition-induced immune-related adverse events
High-dose steroids constitute the cornerstone of first-line treatment for immune-related adverse events (irAEs) associated with immune checkpoint inhibitors, but compromise antitumor immunity. A deeper understanding of irAEs and their response to steroids can contribute to more targeted irAE management regimens. We took a multi-omics approach to identify blood- and tissue-based predictors of steroid response and to explore underlying mechanisms of steroid non-response in irAEs. In the blood, steroid non-response correlated with trends for elevated Tc1/Tc17 CD8 + T cells and serum interleukin (IL)-17, IL-6, IL-12 and IL-23 prior to initiation of steroids, along with persistent (CD8 + ) T cell proliferation and activation after start of steroids. A remarkably fast decrease in inflammatory gene signatures and lymphocyte infiltration was observed in colitis tissue of steroid responders obtained within 24h after initiation of steroids. Peripheral T cell PD-1 receptor occupancy was not associated with steroid response. Colitis tissue of steroid non-responders was enriched for activated CD4 + memory T cells and a pronounced type 1/17 immune response. Together, our findings suggest rapid immunological effects of steroids in circulating cells and irAE-affected tissue and support that an enhanced type 1/type 17 response is associated with steroid non-response in irAEs.
Open AccessBlood and tissue correlates of steroid non-response in checkpoint inhibition-induced immune-related adverse events
High-dose steroids constitute the cornerstone of first-line treatment for immune-related adverse events (irAEs) associated with immune checkpoint inhibitors, but compromise antitumor immunity. A deeper understanding of irAEs and their response to steroids can contribute to more targeted irAE management regimens. We took a multi-omics approach to identify blood- and tissue-based predictors of steroid response and to explore underlying mechanisms of steroid non-response in irAEs. In the blood, steroid non-response correlated with trends for elevated Tc1/Tc17 CD8 + T cells and serum interleukin (IL)-17, IL-6, IL-12 and IL-23 prior to initiation of steroids, along with persistent (CD8 + ) T cell proliferation and activation after start of steroids. A remarkably fast decrease in inflammatory gene signatures and lymphocyte infiltration was observed in colitis tissue of steroid responders obtained within 24h after initiation of steroids. Peripheral T cell PD-1 receptor occupancy was not associated with steroid response. Colitis tissue of steroid non-responders was enriched for activated CD4 + memory T cells and a pronounced type 1/17 immune response. Together, our findings suggest rapid immunological effects of steroids in circulating cells and irAE-affected tissue and support that an enhanced type 1/type 17 response is associated with steroid non-response in irAEs.
- Open Access
A Novel Dataset for Nuclei and Tissue Segmentation in Melanoma with baseline nuclei segmentation and tissue segmentation benchmarks
Background Melanoma is an aggressive form of skin cancer in which tumor-infiltrating lymphocytes (TILs) are a biomarker for recurrence and treatment response. Manual TIL assessment is prone to interobserver variability, and current deep learning models are not publicly accessible or have low performance. Deep learning models, however, have the potential of consistent spatial evaluation of TILs and other immune cell subsets with the potential of improved prognostic and predictive value. To make the development of these models possible, we created the Panoptic Segmentation of nUclei and tissue in advanced MelanomA (PUMA) dataset and assessed the performance of several state-of-the-art deep learning models. In addition, we show how to improve model performance further by using heuristic post-processing in which nuclei classes are updated based on their tissue localization. Results The PUMA dataset includes 155 primary and 155 metastatic melanoma H&E stained regions of interest with nuclei and tissue annotations from a single melanoma referral institution. The Hover-NeXt model, trained on the PUMA dataset, demonstrated the best performance for lymphocyte detection, approaching human interobserver agreement. In addition, heuristic post-processing of deep learning models improve the detection of non-common classes, such as epithelial nuclei. Conclusion The PUMA dataset is the first melanoma specific dataset that can be used to develop melanoma-specific nuclei and tissue segmentation models. These models can, in turn, be used for prognostic and predictive biomarker development. Incorporating tissue and nuclei segmentation is a step towards improved deep learning nuclei segmentation performance. We will use this dataset to organize the PUMA challenge in which the goal is to further improve model performance.
Open AccessA Novel Dataset for Nuclei and Tissue Segmentation in Melanoma with baseline nuclei segmentation and tissue segmentation benchmarks
Background Melanoma is an aggressive form of skin cancer in which tumor-infiltrating lymphocytes (TILs) are a biomarker for recurrence and treatment response. Manual TIL assessment is prone to interobserver variability, and current deep learning models are not publicly accessible or have low performance. Deep learning models, however, have the potential of consistent spatial evaluation of TILs and other immune cell subsets with the potential of improved prognostic and predictive value. To make the development of these models possible, we created the Panoptic Segmentation of nUclei and tissue in advanced MelanomA (PUMA) dataset and assessed the performance of several state-of-the-art deep learning models. In addition, we show how to improve model performance further by using heuristic post-processing in which nuclei classes are updated based on their tissue localization. Results The PUMA dataset includes 155 primary and 155 metastatic melanoma H&E stained regions of interest with nuclei and tissue annotations from a single melanoma referral institution. The Hover-NeXt model, trained on the PUMA dataset, demonstrated the best performance for lymphocyte detection, approaching human interobserver agreement. In addition, heuristic post-processing of deep learning models improve the detection of non-common classes, such as epithelial nuclei. Conclusion The PUMA dataset is the first melanoma specific dataset that can be used to develop melanoma-specific nuclei and tissue segmentation models. These models can, in turn, be used for prognostic and predictive biomarker development. Incorporating tissue and nuclei segmentation is a step towards improved deep learning nuclei segmentation performance. We will use this dataset to organize the PUMA challenge in which the goal is to further improve model performance.