Pixel level deep reinforcement learning for accurate and robust medical image segmentation
Open Access
- 10 March 2025
- journal article
- research article
- Published by Springer Nature in Scientific Reports
- Vol. 15 (1) , 1-18
Abstract
Existing deep learning methods have achieved significant success in medical image segmentation. However, this success largely relies on stacking advanced modules and architectures, which has created a path dependency. This path dependency is unsustainable, as it leads to increasingly larger model parameters and higher deployment costs. To break this path dependency, we introduce deep reinforcement learning to enhance segmentation performance. However, current deep reinforcement learning methods face challenges such as high training cost, independent iterative processes, and high uncertainty of segmentation masks. Consequently, we propose a Pixel-level Deep Reinforcement Learning model with pixel-by-pixel Mask Generation (PixelDRL-MG) for more accurate and robust medical image segmentation. PixelDRL-MG adopts a dynamic iterative update policy, directly segmenting the regions of interest without requiring user interaction or coarse segmentation masks. We propose a Pixel-level Asynchronous Advantage Actor-Critic (PA3C) strategy to treat each pixel as an agent whose state (foreground or background) is iteratively updated through direct actions. Our experiments on two commonly used medical image segmentation datasets demonstrate that PixelDRL-MG achieves more superior segmentation performances than the state-of-the-art segmentation baselines (especially in boundaries) using significantly fewer model parameters. We also conducted detailed ablation studies to enhance understanding and facilitate practical application. Additionally, PixelDRL-MG performs well in low-resource settings (i.e., 50-shot or 100-shot), making it an ideal choice for real-world scenarios.Keywords
Funding Information
- National Natural Science Foundation of China (62276089)
- Natural Science Foundation of Hebei Province (F2024202064)
- Ministry of Human Resources and Social Security (RSTH-2023-135-1)
- Natural Science Foundation of Tianjin Municipality (24JCJQJC00200)
- S&T Program of Hebei (225676163GH)
- Shenzhen Longgang District Innovation and Technology Special Fund (LGKCYLWS2024-27)
- Natural Science Foundation of Chongqing Municipality (CSTB2024NSCQ-MSX0314)
This publication has 37 references indexed in Scilit:
- TransUNet: Rethinking the U-Net architecture design for medical image segmentation through the lens of transformersMedical Image Analysis, 2024
- CerviLearnNet: Advancing cervical cancer diagnosis with reinforcement learning-enhanced convolutional networksHeliyon, 2024
- -Net: Medical image segmentation using efficient and effective deep supervisionComputers in Biology and Medicine, 2023
- Swin-Unet: Unet-Like Pure Transformer for Medical Image Segmentation2023
- Painless and accurate medical image analysis using deep reinforcement learning with task-oriented homogenized automatic pre-processingComputers in Biology and Medicine, 2022
- Multi-modal contrastive mutual learning and pseudo-label re-learning for semi-supervised medical image segmentationMedical Image Analysis, 2022
- Overview of the HECKTOR Challenge at MICCAI 2021: Automatic Head and Neck Tumor Segmentation and Outcome Prediction in PET/CT Images2022
- Association of genomic subtypes of lower-grade gliomas with shape features automatically extracted by a deep learning algorithmComputers in Biology and Medicine, 2019
- UNet++: A Nested U-Net Architecture for Medical Image Segmentation2018
- U-Net: Convolutional Networks for Biomedical Image Segmentation2015