Zig-RiR: Zigzag RWKV-in-RWKV for Efficient Medical Image Segmentation
Top Cited Papers
- 17 April 2025
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Medical Imaging
- Vol. 44 (8) , 3245-3257
Abstract
Medical image segmentation has made significant strides with the development of basic models. Specifically, models that combine CNNs with transformers can successfully extract both local and global features. However, these models inherit the transformer’s quadratic computational complexity, limiting their efficiency. Inspired by the recent Receptance Weighted Key Value (RWKV) model, which achieves linear complexity for long-distance modeling, we explore its potential for medical image segmentation. While directly applying vision-RWKV yields suboptimal results due to insufficient local feature exploration and disrupted spatial continuity, we propose a novel nested structure, Zigzag RWKV-in-RWKV (Zig-RiR), to address these issues. It consists of Outer and Inner RWKV blocks to adeptly capture both global and local features without disrupting spatial continuity. We treat local patches as ”visual sentences” and use the Outer Zig-RWKV to explore global information. Then, we decompose each sentence into subpatches (”visual words”) and use the Inner Zig-RWKV to further explore local information among words, at negligible computational cost. We also introduce a Zigzag-WKV attention mechanism to ensure spatial continuity during token scanning. By aggregating visual word and sentence features, our Zig-RiR can effectively explore both global and local information while preserving spatial continuity. Experiments on four medical image segmentation datasets of both 2D and 3D modalities demonstrate the superior accuracy and efficiency of our method, outperforming the state-of-the-art method 14.4 times in speed and reducing GPU memory usage by 89.5% when testing on 1024 × 1024 high-resolution medical images. Our code is available at https://github.com/txchen-USTC/Zig-RiR.Keywords
Funding Information
- National Natural Science Foundation of China (62472396, 62121002)
- Anhui Provincial Science and Technology Major Project (2023z020006)
- advanced computing resources provided by the Supercomputing Center of the USTC
This publication has 40 references indexed in Scilit:
- ZigMa: A DiT-style Zigzag Mamba Diffusion Model2024
- SegMamba: Long-Range Sequential Modeling Mamba for 3D Medical Image Segmentation2024
- Swin-UMamba: Mamba-Based UNet with ImageNet-Based Pretraining2024
- CTC-Net: A Novel Coupled Feature-Enhanced Transformer and Inverted Convolution Network for Medical Image Segmentation2023
- EGE-UNet: An Efficient Group Enhanced UNet for Skin Lesion Segmentation2023
- Swin-Unet: Unet-Like Pure Transformer for Medical Image Segmentation2023
- PHTrans: Parallelly Aggregating Global and Local Representations for Medical Image Segmentation2022
- UNeXt: MLP-Based Rapid Medical Image Segmentation Network2022
- FCN-Transformer Feature Fusion for Polyp Segmentation2022
- CoTr: Efficiently Bridging CNN and Transformer for 3D Medical Image Segmentation2021